2026-02-11T23:30:06.055+0000 7f21c4239780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:06.055+0000 7f21c4239780 10 monclient: get_monmap_and_config 2026-02-11T23:30:06.055+0000 7f21c4239780 10 monclient: build_initial_monmap 2026-02-11T23:30:06.055+0000 7f21c4239780 10 monclient: monmap: epoch 0 fsid c7e182d9-676d-4eeb-80b1-915a88d421e5 last_changed 2026-02-11T23:30:06.056300+0000 created 2026-02-11T23:30:06.056300+0000 min_mon_release 0 (unknown) election_strategy: 1 0: v1:127.0.0.1:7180/0 mon.noname-a-legacy 1: v2:127.0.0.1:7180/0 mon.noname-a 2026-02-11T23:30:06.056+0000 7f21c4239780 1 Processor -- start 2026-02-11T23:30:06.056+0000 7f21c4239780 1 -- start start 2026-02-11T23:30:06.056+0000 7f21c4239780 10 monclient: init 2026-02-11T23:30:06.056+0000 7f21c4239780 10 monclient: _reopen_session rank -1 2026-02-11T23:30:06.056+0000 7f21c4239780 10 monclient: _add_conns ranks=[1,0] 2026-02-11T23:30:06.056+0000 7f21c4239780 1 --2- >> v2:127.0.0.1:7180/0 conn(0x26e3c00 0x28a0000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:06.056+0000 7f21c4239780 10 monclient(hunting): picked mon.noname-a con 0x26e3c00 addr v2:127.0.0.1:7180/0 2026-02-11T23:30:06.056+0000 7f21c4239780 10 monclient(hunting): picked mon.noname-a-legacy con 0x352c000 addr v1:127.0.0.1:7180/0 2026-02-11T23:30:06.056+0000 7f21c4239780 1 -- --> v1:127.0.0.1:7180/0 -- auth(proto 0 26 bytes epoch 0) -- 0x274cf00 con 0x352c000 2026-02-11T23:30:06.056+0000 7f21c4239780 10 monclient(hunting): start opening mon connection 2026-02-11T23:30:06.056+0000 7f21c4239780 1 -- --> v2:127.0.0.1:7180/0 -- mon_getmap magic: 0 -- 0x2766c80 con 0x26e3c00 2026-02-11T23:30:06.056+0000 7f21c4239780 10 monclient(hunting): _renew_subs 2026-02-11T23:30:06.056+0000 7f21c4239780 10 monclient(hunting): authenticate will time out at 1286.719427s 2026-02-11T23:30:06.056+0000 7f21c19fc640 1 --2- >> v2:127.0.0.1:7180/0 conn(0x26e3c00 0x28a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:06.056+0000 7f21c19fc640 1 --2- >> v2:127.0.0.1:7180/0 conn(0x26e3c00 0x28a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:127.0.0.1:7180/0 says I am v2:127.0.0.1:42228/0 (socket says 127.0.0.1:42228) 2026-02-11T23:30:06.056+0000 7f21c19fc640 1 -- 127.0.0.1:0/2794498904 learned_addr learned my addr 127.0.0.1:0/2794498904 (peer_addr_for_me v2:127.0.0.1:0/0) 2026-02-11T23:30:06.056+0000 7f21c19fc640 10 monclient(hunting): get_auth_request con 0x26e3c00 auth_method 0 2026-02-11T23:30:06.056+0000 7f21c19fc640 10 monclient(hunting): get_auth_request method 1 preferred_modes [1] 2026-02-11T23:30:06.056+0000 7f21c19fc640 10 monclient(hunting): _init_auth method 1 2026-02-11T23:30:06.056+0000 7f21c19fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-11T23:30:06.056+0000 7f21c19fc640 10 monclient(hunting): handle_auth_done global_id 4103 payload 0 2026-02-11T23:30:06.056+0000 7f21c19fc640 1 -- 127.0.0.1:0/2794498904 >> v1:127.0.0.1:7180/0 conn(0x352c000 legacy=0x286d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-11T23:30:06.056+0000 7f21c19fc640 10 monclient: _finish_hunting 0 2026-02-11T23:30:06.056+0000 7f21c19fc640 1 monclient: found mon.noname-a 2026-02-11T23:30:06.056+0000 7f21c19fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:06.056+0000 7f21c19fc640 10 monclient: _send_mon_message to mon.noname-a at v2:127.0.0.1:7180/0 2026-02-11T23:30:06.056+0000 7f21c19fc640 1 -- 127.0.0.1:0/2794498904 --> v2:127.0.0.1:7180/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x2746ea0 con 0x26e3c00 2026-02-11T23:30:06.056+0000 7f21c19fc640 1 --2- 127.0.0.1:0/2794498904 >> v2:127.0.0.1:7180/0 conn(0x26e3c00 0x28a0000 crc :-1 s=READY pgs=7 gs=2 cs=0 l=1 c_cookie=5410645351211202761 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b167947b48ff8c9 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:06.056+0000 7f21c09fa640 1 -- 127.0.0.1:0/2794498904 <== mon.0 v2:127.0.0.1:7180/0 1 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x274adc0 con 0x26e3c00 2026-02-11T23:30:06.056+0000 7f21c09fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:06.056+0000 7f21c09fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-11T23:30:06.056+0000 7f21c09fa640 10 monclient: dump: epoch 1 fsid c7e182d9-676d-4eeb-80b1-915a88d421e5 last_changed 2026-02-11T23:30:05.661167+0000 created 2026-02-11T23:30:05.661167+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:06.056+0000 7f21c09fa640 10 monclient: _finish_auth 0 2026-02-11T23:30:06.056+0000 7f21c09fa640 10 monclient: _check_auth_tickets 2026-02-11T23:30:06.056+0000 7f21c09fa640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:36.057329+0000) 2026-02-11T23:30:06.056+0000 7f21c09fa640 1 -- 127.0.0.1:0/2794498904 <== mon.0 v2:127.0.0.1:7180/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x274cf00 con 0x26e3c00 2026-02-11T23:30:06.056+0000 7f21c09fa640 10 monclient: handle_config config(0 keys) 2026-02-11T23:30:06.056+0000 7f21c09fa640 1 -- 127.0.0.1:0/2794498904 <== mon.0 v2:127.0.0.1:7180/0 3 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x274b080 con 0x26e3c00 2026-02-11T23:30:06.056+0000 7f21c4239780 5 monclient: authenticate success, global_id 4103 2026-02-11T23:30:06.056+0000 7f21c09fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:06.056+0000 7f21c09fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-11T23:30:06.056+0000 7f21c09fa640 10 monclient: dump: epoch 1 fsid c7e182d9-676d-4eeb-80b1-915a88d421e5 last_changed 2026-02-11T23:30:05.661167+0000 created 2026-02-11T23:30:05.661167+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:06.056+0000 7f21c4239780 10 monclient: get_monmap_and_config success 2026-02-11T23:30:06.056+0000 7f21c4239780 4 set_mon_vals no callback set 2026-02-11T23:30:06.057+0000 7f21c4239780 10 monclient: shutdown 2026-02-11T23:30:06.057+0000 7f21c4239780 20 monclient: shutdown discarding 0 pending message(s) 2026-02-11T23:30:06.057+0000 7f21c4239780 1 -- 127.0.0.1:0/2794498904 >> v2:127.0.0.1:7180/0 conn(0x26e3c00 msgr2=0x28a0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-11T23:30:06.057+0000 7f21c4239780 1 --2- 127.0.0.1:0/2794498904 >> v2:127.0.0.1:7180/0 conn(0x26e3c00 0x28a0000 crc :-1 s=READY pgs=7 gs=2 cs=0 l=1 c_cookie=5410645351211202761 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:06.057+0000 7f21c4239780 1 -- 127.0.0.1:0/2794498904 shutdown_connections 2026-02-11T23:30:06.057+0000 7f21c4239780 1 --2- 127.0.0.1:0/2794498904 >> v2:127.0.0.1:7180/0 conn(0x26e3c00 0x28a0000 unknown :-1 s=CLOSED pgs=7 gs=2 cs=0 l=1 c_cookie=5410645351211202761 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:06.057+0000 7f21c4239780 1 -- 127.0.0.1:0/2794498904 >> 127.0.0.1:0/2794498904 conn(0x26e3800 msgr2=0x286c800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-11T23:30:06.057+0000 7f21c4239780 1 -- 127.0.0.1:0/2794498904 shutdown_connections 2026-02-11T23:30:06.057+0000 7f21c4239780 1 -- 127.0.0.1:0/2794498904 wait complete. 2026-02-11T23:30:06.057+0000 7f21c4239780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:06.057+0000 7f21c4239780 0 ceph version 20.3.0-5256-gf2a98edf (f2a98edfc002a9c9c8df236a812719da23bb2dd0) tentacle (dev - RelWithDebInfo), process ceph-osd, pid 314694 2026-02-11T23:30:06.058+0000 7f21c4239780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:06.058+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) mkfs path td/osd-backfill-space/0 2026-02-11T23:30:06.058+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.058+0000 7f21c4239780 -1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open stat got: (1) Operation not permitted 2026-02-11T23:30:06.058+0000 7f21c4239780 -1 bluestore(td/osd-backfill-space/0) _read_fsid unparsable uuid 2026-02-11T23:30:06.058+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) mkfs using provided fsid 37ab9e04-8330-421e-b80d-6f0af31f81ef 2026-02-11T23:30:06.058+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _setup_block_symlink_or_file resized block file to 100 GiB 2026-02-11T23:30:06.058+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.058+0000 7f21c4239780 0 bdev(0x26e2c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.058+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.059+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.059+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.059+0000 7f21c4239780 0 bdev(0x26e3c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.059+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.059+0000 7f21c4239780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:06.059+0000 7f21c4239780 1 bluefs mkfs osd_uuid 37ab9e04-8330-421e-b80d-6f0af31f81ef 2026-02-11T23:30:06.059+0000 7f21c4239780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:06.060+0000 7f21c4239780 1 bluefs mkfs super super(uuid 1819ca90-2e3f-42f6-8ae5-243d9621d726 osd 37ab9e04-8330-421e-b80d-6f0af31f81ef seq 1 block_size 0x1000 log_fnode 0xfile(ino 1 size 0x1000 mtime 0.000000 allocated 400000 alloc_commit 0 extents [1:0x2000~400000])) 2026-02-11T23:30:06.060+0000 7f21c4239780 1 bluefs mount 2026-02-11T23:30:06.060+0000 7f21c4239780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:06.061+0000 7f21c4239780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.061+0000 7f21c4239780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.061+0000 7f21c4239780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.061+0000 7f21c4239780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.061+0000 7f21c4239780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.061+0000 7f21c4239780 1 bluefs mount shared_bdev_used = 4194304 2026-02-11T23:30:06.061+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Git sha 0 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: DB Session ID: T2BXRI51QTPGCVBX1N5Q 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: SST files in db dir, Total Num: 0, files: 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Write Ahead Log file in db.wal: 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.create_if_missing: 1 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.env: 0x27cbf80 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.info_log: 0x27417e0 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:06.062+0000 7f21c4239780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.write_buffer_manager: 0x26dd540 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:06.063+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:317] Creating manifest 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000001 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2741f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c950e167-b477-40c0-82b4-f501f07d8a84 2026-02-11T23:30:06.064+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5047] Creating manifest 5 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x2751180 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: DB pointer 0x2770000 2026-02-11T23:30:06.066+0000 7f21b79e8640 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2026-02-11T23:30:06.066+0000 7f21b79e8640 4 rocksdb: [db/db_impl/db_impl.cc:1111] ** DB Stats ** Uptime(secs): 0.0 total, 0.0 interval Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x26bfe10#314703 capacity: 1.35 GB usage: 0.00 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [default] ** 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2741f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.066+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [m-0] (ID 1) 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2741f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.067+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [m-1] (ID 2) 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2741f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.068+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [m-2] (ID 3) 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2741f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.070+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [p-0] (ID 4) 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2741f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.071+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [p-1] (ID 5) 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2741f20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.073+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [p-2] (ID 6) 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27191a0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bf0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.074+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [O-0] (ID 7) 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27191a0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bf0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.076+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [O-1] (ID 8) 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27191a0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bf0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.078+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [O-2] (ID 9) 2026-02-11T23:30:06.079+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:06.079+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [L] (ID 10) 2026-02-11T23:30:06.081+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:06.081+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:3106] Created column family [P] (ID 11) 2026-02-11T23:30:06.083+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:06.083+0000 7f21c4239780 1 freelist create size 0x1900000000 bytes_per_block 0x1000 blocks 0x1900000 blocks_per_key 0x80 2026-02-11T23:30:06.084+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-11T23:30:06.086+0000 7f21c4239780 1 freelist shutdown 2026-02-11T23:30:06.086+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:06.086+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:06.086+0000 7f21c4239780 1 bluefs umount 2026-02-11T23:30:06.086+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.120+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.150+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.150+0000 7f21c4239780 0 bdev(0x26e2c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.150+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.150+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.186+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.186+0000 7f21c4239780 0 bdev(0x26e2c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.186+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.186+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.186+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.186+0000 7f21c4239780 0 bdev(0x26e3c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.186+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.186+0000 7f21c4239780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:06.186+0000 7f21c4239780 1 bluefs mount 2026-02-11T23:30:06.186+0000 7f21c4239780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:06.187+0000 7f21c4239780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.187+0000 7f21c4239780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.187+0000 7f21c4239780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.187+0000 7f21c4239780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.187+0000 7f21c4239780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.187+0000 7f21c4239780 1 bluefs mount shared_bdev_used = 27197440 2026-02-11T23:30:06.187+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Git sha 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: DB Session ID: T2BXRI51QTPGCVBX1N5R 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 605 Bytes 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: SST files in db dir, Total Num: 0, files: 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Write Ahead Log file in db.wal: 000004.log size: 276 ; 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.env: 0x35203f0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.info_log: 0x274e020 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.write_buffer_manager: 0x26dd540 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_readonly.cc:25] Opening the db in read only mode 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000005 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2876300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2876300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.188+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2876300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2876300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2876300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2876300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2876300) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2719520) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bf0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2719520) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bf0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2719520) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bf0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:06.189+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 29, last_sequence is 0, log_number is 4,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 0 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 4 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 4 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 4 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 4 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 4 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 4 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 4 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 4 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 4 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 4 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 4 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c950e167-b477-40c0-82b4-f501f07d8a84 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852606193938, "job": 1, "event": "recovery_started", "wal_files": [4]} 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2026-02-11T23:30:06.193+0000 7f21c4239780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852606194233, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:06.193+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:06.193+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old nid_max 0 2026-02-11T23:30:06.193+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old blobid_max 0 2026-02-11T23:30:06.193+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-11T23:30:06.193+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_super_meta min_alloc_size 0x1000 2026-02-11T23:30:06.194+0000 7f21c4239780 1 freelist init 2026-02-11T23:30:06.194+0000 7f21c4239780 1 freelist _read_cfg 2026-02-11T23:30:06.194+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-11T23:30:06.194+0000 7f21c4239780 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2026-02-11T23:30:06.194+0000 7f21c4239780 1 fbmap_alloc 0x282c360 shutdown 2026-02-11T23:30:06.194+0000 7f21c4239780 0 bluestore(td/osd-backfill-space/0) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2026-02-11T23:30:06.195+0000 7f21c4239780 1 bluestore::NCB::read_allocation_from_drive_on_startup::::Allocation Recovery was completed in 0.000194 seconds, extent_count=1 2026-02-11T23:30:06.195+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _init_alloc loaded 0 B in 0 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-11T23:30:06.195+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:06.195+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:06.195+0000 7f21c4239780 1 bluefs umount 2026-02-11T23:30:06.195+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.226+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.226+0000 7f21c4239780 0 bdev(0x26e3c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.226+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.226+0000 7f21c4239780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:06.226+0000 7f21c4239780 1 bluefs mount 2026-02-11T23:30:06.226+0000 7f21c4239780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:06.227+0000 7f21c4239780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.227+0000 7f21c4239780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.227+0000 7f21c4239780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.227+0000 7f21c4239780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.227+0000 7f21c4239780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.227+0000 7f21c4239780 1 bluefs mount shared_bdev_used = 27197440 2026-02-11T23:30:06.227+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Git sha 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: DB Session ID: T2BXRI51QTPGCVBX1N5O 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 605 Bytes 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: SST files in db dir, Total Num: 0, files: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Write Ahead Log file in db.wal: 000004.log size: 276 ; 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.env: 0x26fe7e0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.info_log: 0x2741660 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_buffer_manager: 0x26dd540 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_readonly.cc:25] Opening the db in read only mode 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000005 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27418c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27418c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27418c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27418c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27418c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27418c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27418c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bfe10 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27c0340) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bf0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27c0340) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bf0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x27c0340) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x26bf0f0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:06.228+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 29, last_sequence is 0, log_number is 4,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 0 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 4 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 4 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 4 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 4 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 4 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 4 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 4 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 4 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 4 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 4 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 4 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c950e167-b477-40c0-82b4-f501f07d8a84 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852606233247, "job": 1, "event": "recovery_started", "wal_files": [4]} 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2026-02-11T23:30:06.232+0000 7f21c4239780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852606233387, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:06.232+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:06.233+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open <<>> check (regular) start sb_tracker_hash_size:134217728 2026-02-11T23:30:06.282+0000 7f21c4239780 1 bluefs fsck 2026-02-11T23:30:06.282+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open checking shared_blobs (phase 1) 2026-02-11T23:30:06.282+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open walking object keyspace 2026-02-11T23:30:06.284+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open checking shared_blobs (phase 2) 2026-02-11T23:30:06.294+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open checking pool_statfs 2026-02-11T23:30:06.294+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open checking for stray omap data 2026-02-11T23:30:06.294+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open checking deferred events 2026-02-11T23:30:06.294+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _fsck_on_open <<>> with 0 errors, 0 warnings, 0 repaired, 0 remaining in 0.013353 seconds 2026-02-11T23:30:06.294+0000 7f21b8100640 0 bluestore(td/osd-backfill-space/0) allocation stats probe 0: cnt: 0 frags: 0 size: 0 2026-02-11T23:30:06.294+0000 7f21b8100640 0 bluestore(td/osd-backfill-space/0) probe -1: 0, 0, 0 2026-02-11T23:30:06.294+0000 7f21b8100640 0 bluestore(td/osd-backfill-space/0) probe -2: 0, 0, 0 2026-02-11T23:30:06.294+0000 7f21b8100640 0 bluestore(td/osd-backfill-space/0) probe -4: 0, 0, 0 2026-02-11T23:30:06.294+0000 7f21b8100640 0 bluestore(td/osd-backfill-space/0) probe -8: 0, 0, 0 2026-02-11T23:30:06.294+0000 7f21b8100640 0 bluestore(td/osd-backfill-space/0) probe -16: 0, 0, 0 2026-02-11T23:30:06.294+0000 7f21b8100640 0 bluestore(td/osd-backfill-space/0) ------------ 2026-02-11T23:30:06.294+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:06.294+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:06.294+0000 7f21c4239780 1 bluefs umount 2026-02-11T23:30:06.294+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.316+0000 7f21c4239780 1 freelist shutdown 2026-02-11T23:30:06.316+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.332+0000 7f21c4239780 0 bluestore(td/osd-backfill-space/0) mkfs success 2026-02-11T23:30:06.332+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.332+0000 7f21c4239780 0 bdev(0x26e2c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.332+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.332+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.346+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.346+0000 7f21c4239780 0 bdev(0x26e2c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.346+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.346+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.362+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.362+0000 7f21c4239780 0 bdev(0x26e2c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.362+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.362+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.362+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.362+0000 7f21c4239780 0 bdev(0x26e3c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.362+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.362+0000 7f21c4239780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:06.362+0000 7f21c4239780 1 bluefs mount 2026-02-11T23:30:06.362+0000 7f21c4239780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:06.363+0000 7f21c4239780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.363+0000 7f21c4239780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.363+0000 7f21c4239780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.363+0000 7f21c4239780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.363+0000 7f21c4239780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.363+0000 7f21c4239780 1 bluefs mount shared_bdev_used = 27197440 2026-02-11T23:30:06.363+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:06.364+0000 7f21c4239780 1 rocksdb: SocketHook cannot register SocketHook 2026-02-11T23:30:06.364+0000 7f21c4239780 1 rocksdb: SocketHook cannot register SocketHook 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Git sha 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: DB Session ID: T2BXRI51QTPGCVBX1N5P 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 605 Bytes 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: SST files in db dir, Total Num: 0, files: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Write Ahead Log file in db.wal: 000004.log size: 276 ; 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.env: 0x27cbf10 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.info_log: 0x2740360 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_buffer_manager: 0x26ddea0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_readonly.cc:25] Opening the db in read only mode 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000005 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x26f3ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367cd30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x26f3ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367cd30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x26f3ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367cd30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x26f3ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367cd30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x26f3ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367cd30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x26f3ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367cd30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x26f3ec0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367cd30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2730b20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367c790 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2730b20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367c790 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2730b20) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367c790 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.364+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:06.365+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 29, last_sequence is 0, log_number is 4,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 0 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 4 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 4 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 4 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 4 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 4 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 4 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 4 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 4 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 4 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 4 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 4 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c950e167-b477-40c0-82b4-f501f07d8a84 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852606369609, "job": 1, "event": "recovery_started", "wal_files": [4]} 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852606369762, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:06.369+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:06.369+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old nid_max 0 2026-02-11T23:30:06.369+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old blobid_max 0 2026-02-11T23:30:06.369+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-11T23:30:06.369+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_super_meta min_alloc_size 0x1000 2026-02-11T23:30:06.369+0000 7f21c4239780 1 freelist init 2026-02-11T23:30:06.369+0000 7f21c4239780 1 freelist _read_cfg 2026-02-11T23:30:06.369+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-11T23:30:06.369+0000 7f21c4239780 1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2026-02-11T23:30:06.369+0000 7f21c4239780 1 fbmap_alloc 0x282dc20 shutdown 2026-02-11T23:30:06.369+0000 7f21c4239780 0 bluestore(td/osd-backfill-space/0) _init_alloc::NCB::restore_allocator() failed! Run Full Recovery from ONodes (might take a while) ... 2026-02-11T23:30:06.369+0000 7f21c4239780 1 bluestore::NCB::read_allocation_from_drive_on_startup::::Allocation Recovery was completed in 0.000186 seconds, extent_count=1 2026-02-11T23:30:06.369+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _init_alloc loaded 0 B in 0 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffe000, fragmentation 0 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:06.369+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:06.369+0000 7f21c4239780 1 bluefs umount 2026-02-11T23:30:06.369+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.395+0000 7f21c4239780 0 bdev(0x26e3c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bluefs mount 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bluefs mount shared_bdev_used = 27197440 2026-02-11T23:30:06.395+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:06.396+0000 7f21c4239780 1 rocksdb: SocketHook cannot register SocketHook 2026-02-11T23:30:06.396+0000 7f21c4239780 1 rocksdb: SocketHook cannot register SocketHook 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Git sha 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: DB Session ID: T2BXRI51QTPGCVBX1N5M 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 605 Bytes 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: SST files in db dir, Total Num: 0, files: 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Write Ahead Log file in db.wal: 000004.log size: 276 ; 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.env: 0x26fe9a0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.info_log: 0x2741660 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.write_buffer_manager: 0x26ddea0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000005 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x274ec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367c3d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x274ec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367c3d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.396+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x274ec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367c3d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x274ec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367c3d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x274ec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367c3d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x274ec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367c3d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x274ec40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367c3d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x275d1c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367cd30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x275d1c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367cd30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x275d1c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x367cd30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:06.397+0000 7f21c4239780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 29, last_sequence is 0, log_number is 4,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 0 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 4 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 4 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 4 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 4 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 4 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 4 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 4 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 4 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 4 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 4 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 4 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c950e167-b477-40c0-82b4-f501f07d8a84 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852606402068, "job": 1, "event": "recovery_started", "wal_files": [4]} 2026-02-11T23:30:06.401+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2026-02-11T23:30:06.402+0000 7f21c4239780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852606402542, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 1412, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 11, "table_properties": {"data_size": 262, "index_size": 33, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 243, "raw_average_key_size": 22, "raw_value_size": 69, "raw_average_value_size": 6, "num_data_blocks": 1, "num_entries": 11, "num_filter_entries": 11, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": ".T:int64_array.b:bitwise_xor", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "LZ4", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1770852606, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c950e167-b477-40c0-82b4-f501f07d8a84", "db_session_id": "T2BXRI51QTPGCVBX1N5M", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2026-02-11T23:30:06.402+0000 7f21c4239780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852606402578, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:06.402+0000 7f21c4239780 4 rocksdb: [db/version_set.cc:5047] Creating manifest 32 2026-02-11T23:30:06.404+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x3497500 2026-02-11T23:30:06.404+0000 7f21c4239780 4 rocksdb: DB pointer 0x2770000 2026-02-11T23:30:06.404+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:06.405+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _upgrade_super from 4, latest 4 2026-02-11T23:30:06.405+0000 7f21c4239780 1 bluestore(td/osd-backfill-space/0) _upgrade_super done 2026-02-11T23:30:06.405+0000 7f21c4239780 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e0 maps [] lci=[0,0] tlb=0) 2026-02-11T23:30:06.414+0000 7f21be90d640 0 bluestore(td/osd-backfill-space/0) allocation stats probe 1: cnt: 0 frags: 0 size: 0 2026-02-11T23:30:06.414+0000 7f21be90d640 0 bluestore(td/osd-backfill-space/0) probe -1: 0, 0, 0 2026-02-11T23:30:06.414+0000 7f21be90d640 0 bluestore(td/osd-backfill-space/0) probe -3: 0, 0, 0 2026-02-11T23:30:06.414+0000 7f21be90d640 0 bluestore(td/osd-backfill-space/0) probe -5: 0, 0, 0 2026-02-11T23:30:06.414+0000 7f21be90d640 0 bluestore(td/osd-backfill-space/0) probe -9: 0, 0, 0 2026-02-11T23:30:06.414+0000 7f21be90d640 0 bluestore(td/osd-backfill-space/0) probe -17: 0, 0, 0 2026-02-11T23:30:06.414+0000 7f21be90d640 0 bluestore(td/osd-backfill-space/0) ------------ 2026-02-11T23:30:06.414+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:06.415+0000 7f21c4239780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:06.417+0000 7f21c4239780 1 fbmap_alloc 0x282d7a0 shutdown 2026-02-11T23:30:06.417+0000 7f21c4239780 1 bluefs umount 2026-02-11T23:30:06.417+0000 7f21c4239780 1 bdev(0x26e3c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.456+0000 7f21c4239780 1 freelist shutdown 2026-02-11T23:30:06.456+0000 7f21c4239780 1 bdev(0x26e2c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.480+0000 7f21c4239780 0 created object store td/osd-backfill-space/0 for osd.0 fsid c7e182d9-676d-4eeb-80b1-915a88d421e5 2026-02-11T23:30:06.611+0000 7fac73829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:06.611+0000 7fac73829780 10 monclient: get_monmap_and_config 2026-02-11T23:30:06.611+0000 7fac73829780 10 monclient: build_initial_monmap 2026-02-11T23:30:06.611+0000 7fac73829780 10 monclient: monmap: epoch 0 fsid c7e182d9-676d-4eeb-80b1-915a88d421e5 last_changed 2026-02-11T23:30:06.612337+0000 created 2026-02-11T23:30:06.612337+0000 min_mon_release 0 (unknown) election_strategy: 1 0: v1:127.0.0.1:7180/0 mon.noname-a-legacy 1: v2:127.0.0.1:7180/0 mon.noname-a 2026-02-11T23:30:06.612+0000 7fac73829780 1 Processor -- start 2026-02-11T23:30:06.612+0000 7fac73829780 1 -- start start 2026-02-11T23:30:06.612+0000 7fac73829780 10 monclient: init 2026-02-11T23:30:06.612+0000 7fac73829780 10 monclient: _reopen_session rank -1 2026-02-11T23:30:06.612+0000 7fac73829780 10 monclient: _add_conns ranks=[1,0] 2026-02-11T23:30:06.612+0000 7fac73829780 1 --2- >> v2:127.0.0.1:7180/0 conn(0x2519c00 0x26d6000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:06.612+0000 7fac73829780 10 monclient(hunting): picked mon.noname-a con 0x2519c00 addr v2:127.0.0.1:7180/0 2026-02-11T23:30:06.612+0000 7fac73829780 10 monclient(hunting): picked mon.noname-a-legacy con 0x3362000 addr v1:127.0.0.1:7180/0 2026-02-11T23:30:06.612+0000 7fac73829780 1 -- --> v1:127.0.0.1:7180/0 -- auth(proto 0 26 bytes epoch 0) -- 0x2582f00 con 0x3362000 2026-02-11T23:30:06.612+0000 7fac73829780 10 monclient(hunting): start opening mon connection 2026-02-11T23:30:06.612+0000 7fac73829780 1 -- --> v2:127.0.0.1:7180/0 -- mon_getmap magic: 0 -- 0x259d180 con 0x2519c00 2026-02-11T23:30:06.612+0000 7fac73829780 10 monclient(hunting): _renew_subs 2026-02-11T23:30:06.612+0000 7fac73829780 10 monclient(hunting): authenticate will time out at 1287.275478s 2026-02-11T23:30:06.612+0000 7fac713fc640 1 --2- >> v2:127.0.0.1:7180/0 conn(0x2519c00 0x26d6000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:06.612+0000 7fac713fc640 1 --2- >> v2:127.0.0.1:7180/0 conn(0x2519c00 0x26d6000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:127.0.0.1:7180/0 says I am v2:127.0.0.1:42250/0 (socket says 127.0.0.1:42250) 2026-02-11T23:30:06.612+0000 7fac713fc640 1 -- 127.0.0.1:0/2387714939 learned_addr learned my addr 127.0.0.1:0/2387714939 (peer_addr_for_me v2:127.0.0.1:0/0) 2026-02-11T23:30:06.612+0000 7fac713fc640 10 monclient(hunting): get_auth_request con 0x2519c00 auth_method 0 2026-02-11T23:30:06.612+0000 7fac713fc640 10 monclient(hunting): get_auth_request method 1 preferred_modes [1] 2026-02-11T23:30:06.612+0000 7fac713fc640 10 monclient(hunting): _init_auth method 1 2026-02-11T23:30:06.612+0000 7fac713fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-11T23:30:06.612+0000 7fac713fc640 10 monclient(hunting): handle_auth_done global_id 4106 payload 0 2026-02-11T23:30:06.612+0000 7fac713fc640 1 -- 127.0.0.1:0/2387714939 >> v1:127.0.0.1:7180/0 conn(0x3362000 legacy=0x26a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-11T23:30:06.612+0000 7fac713fc640 10 monclient: _finish_hunting 0 2026-02-11T23:30:06.612+0000 7fac713fc640 1 monclient: found mon.noname-a 2026-02-11T23:30:06.612+0000 7fac713fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:06.612+0000 7fac713fc640 10 monclient: _send_mon_message to mon.noname-a at v2:127.0.0.1:7180/0 2026-02-11T23:30:06.612+0000 7fac713fc640 1 -- 127.0.0.1:0/2387714939 --> v2:127.0.0.1:7180/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x257cea0 con 0x2519c00 2026-02-11T23:30:06.612+0000 7fac713fc640 1 --2- 127.0.0.1:0/2387714939 >> v2:127.0.0.1:7180/0 conn(0x2519c00 0x26d6000 crc :-1 s=READY pgs=10 gs=2 cs=0 l=1 c_cookie=14707845129314072587 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1cc3383839780b server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:06.612+0000 7fac703fa640 1 -- 127.0.0.1:0/2387714939 <== mon.0 v2:127.0.0.1:7180/0 1 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x2580dc0 con 0x2519c00 2026-02-11T23:30:06.612+0000 7fac703fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:06.612+0000 7fac703fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-11T23:30:06.612+0000 7fac703fa640 10 monclient: dump: epoch 1 fsid c7e182d9-676d-4eeb-80b1-915a88d421e5 last_changed 2026-02-11T23:30:05.661167+0000 created 2026-02-11T23:30:05.661167+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:06.612+0000 7fac703fa640 10 monclient: _finish_auth 0 2026-02-11T23:30:06.612+0000 7fac703fa640 10 monclient: _check_auth_tickets 2026-02-11T23:30:06.612+0000 7fac703fa640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:36.613328+0000) 2026-02-11T23:30:06.612+0000 7fac703fa640 1 -- 127.0.0.1:0/2387714939 <== mon.0 v2:127.0.0.1:7180/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x2582f00 con 0x2519c00 2026-02-11T23:30:06.612+0000 7fac73829780 5 monclient: authenticate success, global_id 4106 2026-02-11T23:30:06.612+0000 7fac73829780 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-11T23:30:06.612+0000 7fac703fa640 10 monclient: handle_config config(0 keys) 2026-02-11T23:30:06.612+0000 7fac703fa640 1 -- 127.0.0.1:0/2387714939 <== mon.0 v2:127.0.0.1:7180/0 3 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x2581080 con 0x2519c00 2026-02-11T23:30:06.612+0000 7fac73829780 10 monclient: get_monmap_and_config success 2026-02-11T23:30:06.612+0000 7fac703fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:06.612+0000 7fac73829780 4 set_mon_vals no callback set 2026-02-11T23:30:06.612+0000 7fac703fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-11T23:30:06.612+0000 7fac703fa640 10 monclient: dump: epoch 1 fsid c7e182d9-676d-4eeb-80b1-915a88d421e5 last_changed 2026-02-11T23:30:05.661167+0000 created 2026-02-11T23:30:05.661167+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:06.613+0000 7fac73829780 10 monclient: shutdown 2026-02-11T23:30:06.613+0000 7fac73829780 20 monclient: shutdown discarding 0 pending message(s) 2026-02-11T23:30:06.613+0000 7fac73829780 1 -- 127.0.0.1:0/2387714939 >> v2:127.0.0.1:7180/0 conn(0x2519c00 msgr2=0x26d6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-11T23:30:06.613+0000 7fac73829780 1 --2- 127.0.0.1:0/2387714939 >> v2:127.0.0.1:7180/0 conn(0x2519c00 0x26d6000 crc :-1 s=READY pgs=10 gs=2 cs=0 l=1 c_cookie=14707845129314072587 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:06.613+0000 7fac73829780 1 -- 127.0.0.1:0/2387714939 shutdown_connections 2026-02-11T23:30:06.613+0000 7fac73829780 1 --2- 127.0.0.1:0/2387714939 >> v2:127.0.0.1:7180/0 conn(0x2519c00 0x26d6000 unknown :-1 s=CLOSED pgs=10 gs=2 cs=0 l=1 c_cookie=14707845129314072587 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:06.613+0000 7fac73829780 1 -- 127.0.0.1:0/2387714939 >> 127.0.0.1:0/2387714939 conn(0x2519800 msgr2=0x26a2800 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-11T23:30:06.613+0000 7fac73829780 1 -- 127.0.0.1:0/2387714939 shutdown_connections 2026-02-11T23:30:06.613+0000 7fac73829780 1 -- 127.0.0.1:0/2387714939 wait complete. 2026-02-11T23:30:06.613+0000 7fac73829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:06.613+0000 7fac73829780 0 ceph version 20.3.0-5256-gf2a98edf (f2a98edfc002a9c9c8df236a812719da23bb2dd0) tentacle (dev - RelWithDebInfo), process ceph-osd, pid 315489 2026-02-11T23:30:06.614+0000 7fac73829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-11T23:30:06.614+0000 7fac73829780 5 object store type is bluestore 2026-02-11T23:30:06.614+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.614+0000 7fac73829780 0 bdev(0x2518c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.614+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.615+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.641+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.641+0000 7fac73829780 0 bdev(0x2518c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.641+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.641+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.672+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.672+0000 7fac73829780 0 bdev(0x2518c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.672+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.672+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.703+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.703+0000 7fac73829780 0 bdev(0x2518c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.703+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.703+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.737+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.737+0000 7fac73829780 0 bdev(0x2518c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.737+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.737+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.769+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.769+0000 7fac73829780 0 bdev(0x2518c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.769+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.769+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.769+0000 7fac73829780 1 bdev(0x2519c00 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.769+0000 7fac73829780 0 bdev(0x2519c00 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.769+0000 7fac73829780 1 bdev(0x2519c00 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.769+0000 7fac73829780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:06.769+0000 7fac73829780 1 bdev(0x2519c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.797+0000 7fac73829780 1 bdev(0x2518c00 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.817+0000 7fac73829780 1 objectstore numa_node 0 2026-02-11T23:30:06.817+0000 7fac73829780 0 starting osd.0 osd_data td/osd-backfill-space/0 td/osd-backfill-space/0/journal 2026-02-11T23:30:06.817+0000 7fac73829780 10 picked public_addrs [v2:0.0.0.0:0/0,v1:0.0.0.0:0/0] 2026-02-11T23:30:06.817+0000 7fac73829780 10 there is no public_bind_addrs, defaulting to public_addrs 2026-02-11T23:30:06.817+0000 7fac73829780 -1 Falling back to public interface 2026-02-11T23:30:06.818+0000 7fac73829780 1 -- [v2:0.0.0.0:6800/460666604,v1:0.0.0.0:6801/460666604] _finish_bind bind my_addrs is [v2:0.0.0.0:6800/460666604,v1:0.0.0.0:6801/460666604] 2026-02-11T23:30:06.818+0000 7fac73829780 1 -- [v2:0.0.0.0:6802/460666604,v1:0.0.0.0:6803/460666604] _finish_bind bind my_addrs is [v2:0.0.0.0:6802/460666604,v1:0.0.0.0:6803/460666604] 2026-02-11T23:30:06.818+0000 7fac73829780 1 -- [v2:0.0.0.0:6804/460666604,v1:0.0.0.0:6805/460666604] _finish_bind bind my_addrs is [v2:0.0.0.0:6804/460666604,v1:0.0.0.0:6805/460666604] 2026-02-11T23:30:06.818+0000 7fac73829780 1 -- [v2:0.0.0.0:6806/460666604,v1:0.0.0.0:6807/460666604] _finish_bind bind my_addrs is [v2:0.0.0.0:6806/460666604,v1:0.0.0.0:6807/460666604] 2026-02-11T23:30:06.818+0000 7fac73829780 10 monclient: build_initial_monmap 2026-02-11T23:30:06.818+0000 7fac73829780 10 monclient: monmap: epoch 0 fsid 00000000-0000-0000-0000-000000000000 last_changed 0.000000 created 0.000000 min_mon_release 0 (unknown) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.noname-a 2026-02-11T23:30:06.820+0000 7fac73829780 0 load: jerasure load: lrc 2026-02-11T23:30:06.820+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.820+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.820+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.820+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.820+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.845+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.845+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.845+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.845+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.845+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.865+0000 7fac73829780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:06.865+0000 7fac73829780 0 osd.0:0.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:06.865+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.865+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.865+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.865+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.865+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.889+0000 7fac73829780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:06.889+0000 7fac73829780 0 osd.0:1.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:06.889+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.889+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.889+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.889+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.889+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.913+0000 7fac73829780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:06.913+0000 7fac73829780 0 osd.0:2.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:06.913+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.913+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.913+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.913+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.913+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.937+0000 7fac73829780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:06.937+0000 7fac73829780 0 osd.0:3.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:06.937+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.937+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.937+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.937+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.937+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.961+0000 7fac73829780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:06.961+0000 7fac73829780 0 osd.0:4.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:06.961+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.961+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.961+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.961+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.961+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:06.981+0000 7fac73829780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:06.981+0000 7fac73829780 0 osd.0:5.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:06.981+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:06.981+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:06.981+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:06.981+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:06.981+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:07.009+0000 7fac73829780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:07.009+0000 7fac73829780 0 osd.0:6.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:07.009+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:07.009+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:07.009+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:07.009+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:07.009+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:07.037+0000 7fac73829780 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 58525.17 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:07.037+0000 7fac73829780 0 osd.0:7.OSDShard using op scheduler mclock_scheduler, cutoff=196 2026-02-11T23:30:07.037+0000 7fac73829780 1 -- [v2:0.0.0.0:6800/460666604,v1:0.0.0.0:6801/460666604] start start 2026-02-11T23:30:07.037+0000 7fac73829780 1 -- start start 2026-02-11T23:30:07.037+0000 7fac73829780 1 -- start start 2026-02-11T23:30:07.037+0000 7fac73829780 1 -- [v2:0.0.0.0:6804/460666604,v1:0.0.0.0:6805/460666604] start start 2026-02-11T23:30:07.037+0000 7fac73829780 1 -- [v2:0.0.0.0:6806/460666604,v1:0.0.0.0:6807/460666604] start start 2026-02-11T23:30:07.037+0000 7fac73829780 1 -- [v2:0.0.0.0:6802/460666604,v1:0.0.0.0:6803/460666604] start start 2026-02-11T23:30:07.037+0000 7fac73829780 1 -- start start 2026-02-11T23:30:07.037+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:07.037+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:07.037+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:07.037+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:07.065+0000 7fac73829780 2 osd.0 0 init td/osd-backfill-space/0 (looks like ssd) 2026-02-11T23:30:07.065+0000 7fac73829780 2 osd.0 0 journal td/osd-backfill-space/0/journal 2026-02-11T23:30:07.065+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:07.065+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:07.065+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:07.065+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:07.089+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:07.089+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:07.089+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:07.089+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) close 2026-02-11T23:30:07.109+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:07.109+0000 7fac73829780 0 bdev(0x25e7400 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:07.109+0000 7fac73829780 1 bdev(0x25e7400 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:07.109+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-11T23:30:07.109+0000 7fac73829780 1 bdev(0x34c0800 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:07.109+0000 7fac73829780 0 bdev(0x34c0800 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:07.109+0000 7fac73829780 1 bdev(0x34c0800 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:07.109+0000 7fac73829780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:07.109+0000 7fac73829780 1 bluefs mount 2026-02-11T23:30:07.109+0000 7fac73829780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:07.112+0000 7fac73829780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:07.112+0000 7fac73829780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:07.112+0000 7fac73829780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:07.112+0000 7fac73829780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:07.112+0000 7fac73829780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:07.112+0000 7fac73829780 1 bluefs mount shared_bdev_used = 0 2026-02-11T23:30:07.112+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Git sha 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: DB Session ID: GFOOOOZ5B5960W5C3D62 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: MANIFEST file: MANIFEST-000032 size: 1012 Bytes 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: SST files in db dir, Total Num: 1, files: 000030.sst 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Write Ahead Log file in db.wal: 000031.log size: 950 ; 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.env: 0x25349a0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.info_log: 0x25f60a0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.write_buffer_manager: 0x25c6dc0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: [db/db_impl/db_impl_readonly.cc:25] Opening the db in read only mode 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000032 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2566240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.113+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2566240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2566240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2566240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2566240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2566240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2566240) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4970 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2572c40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2572c40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x2572c40) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:07.114+0000 7fac73829780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000032 succeeded,manifest_file_number is 32, next_file_number is 34, last_sequence is 11, log_number is 5,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 5 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c950e167-b477-40c0-82b4-f501f07d8a84 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852607119087, "job": 1, "event": "recovery_started", "wal_files": [31]} 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2026-02-11T23:30:07.118+0000 7fac73829780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852607119239, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:07.118+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:07.118+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old nid_max 1025 2026-02-11T23:30:07.118+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _open_super_meta old blobid_max 10240 2026-02-11T23:30:07.118+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-11T23:30:07.118+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _open_super_meta min_alloc_size 0x1000 2026-02-11T23:30:07.119+0000 7fac73829780 1 freelist init 2026-02-11T23:30:07.119+0000 7fac73829780 1 freelist _read_cfg 2026-02-11T23:30:07.119+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _open_fm effective freelist_type = null, freelist_alloc_size = 0x1000, min_alloc_size = 0x1000 2026-02-11T23:30:07.121+0000 7fac73829780 1 fbmap_alloc 0x2663200 shutdown 2026-02-11T23:30:07.121+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _init_alloc loaded 100 GiB in 2 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18ffffd000, fragmentation 3.8e-08 2026-02-11T23:30:07.121+0000 7fac73829780 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2026-02-11T23:30:07.121+0000 7fac73829780 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2026-02-11T23:30:07.121+0000 7fac73829780 1 bluefs umount 2026-02-11T23:30:07.121+0000 7fac73829780 1 bdev(0x34c0800 td/osd-backfill-space/0/block) close 2026-02-11T23:30:07.154+0000 7fac73829780 1 bdev(0x34c0800 td/osd-backfill-space/0/block) open path td/osd-backfill-space/0/block 2026-02-11T23:30:07.154+0000 7fac73829780 0 bdev(0x34c0800 td/osd-backfill-space/0/block) ioctl(F_SET_FILE_RW_HINT) on td/osd-backfill-space/0/block failed: (22) Invalid argument 2026-02-11T23:30:07.154+0000 7fac73829780 1 bdev(0x34c0800 td/osd-backfill-space/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-11T23:30:07.154+0000 7fac73829780 1 bluefs add_block_device bdev 1 path td/osd-backfill-space/0/block size 100 GiB 2026-02-11T23:30:07.154+0000 7fac73829780 1 bluefs mount 2026-02-11T23:30:07.154+0000 7fac73829780 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-11T23:30:07.155+0000 7fac73829780 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:07.155+0000 7fac73829780 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:07.155+0000 7fac73829780 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:07.155+0000 7fac73829780 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:07.155+0000 7fac73829780 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-11T23:30:07.155+0000 7fac73829780 1 bluefs mount shared_bdev_used = 27459584 2026-02-11T23:30:07.155+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: RocksDB version: 7.9.2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Git sha 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Compile date 2026-02-11 10:30:18 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: DB SUMMARY 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: DB Session ID: GFOOOOZ5B5960W5C3D63 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: CURRENT file: CURRENT 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: IDENTITY file: IDENTITY 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: MANIFEST file: MANIFEST-000032 size: 1012 Bytes 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: SST files in db dir, Total Num: 1, files: 000030.sst 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: SST files in db.slow dir, Total Num: 0, files: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Write Ahead Log file in db.wal: 000031.log size: 950 ; 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.error_if_exists: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.create_if_missing: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.paranoid_checks: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.flush_verify_memtable_count: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.env: 0x3532000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.fs: LegacyFileSystem 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.info_log: 0x254fe00 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_file_opening_threads: 16 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.statistics: (nil) 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.use_fsync: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_log_file_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.log_file_time_to_roll: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.keep_log_file_num: 1000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.recycle_log_file_num: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.allow_fallocate: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.allow_mmap_reads: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.allow_mmap_writes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.use_direct_reads: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.create_missing_column_families: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.db_log_dir: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.wal_dir: db.wal 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_cache_numshardbits: 6 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.WAL_ttl_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.WAL_size_limit_MB: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.is_fd_close_on_exec: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.advise_random_on_open: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.db_write_buffer_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_buffer_manager: 0x25c7900 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.use_adaptive_mutex: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.rate_limiter: (nil) 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.wal_recovery_mode: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_thread_tracking: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_pipelined_write: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.unordered_write: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.row_cache: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.wal_filter: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.allow_ingest_behind: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.two_write_queues: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.manual_wal_flush: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.wal_compression: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.atomic_flush: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.persist_stats_to_disk: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_dbid_to_manifest: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.log_readahead_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.best_efforts_recovery: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.allow_data_in_errors: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.db_host_id: __hostname__ 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enforce_single_del_contracts: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_background_jobs: 4 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_background_compactions: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_subcompactions: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.writable_file_max_buffer_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.delayed_write_rate : 16777216 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_total_wal_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.stats_dump_period_sec: 600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.stats_persist_period_sec: 600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_open_files: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bytes_per_sync: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.wal_bytes_per_sync: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.strict_bytes_per_sync: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_readahead_size: 2097152 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_background_flushes: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Compression algorithms supported: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: kZSTD supported: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: kXpressCompression supported: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: kBZip2Compression supported: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: kLZ4Compression supported: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: kZlibCompression supported: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: kLZ4HCCompression supported: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: kSnappyCompression supported: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Fast CRC32 supported: Supported on x86 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: DMutex implementation: pthread_mutex_t 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: db/MANIFEST-000032 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.merge_operator: .T:int64_array.b:bitwise_xor 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x38f02c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-0]: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x38f02c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-1]: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x38f02c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [m-2]: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x38f02c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-0]: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x38f02c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-1]: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x38f02c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [p-2]: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x38f02c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f4d30 block_cache_name: BinnedLRUCache block_cache_options: capacity : 1449551462 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-0]: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x38f03c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f52d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-1]: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x38f03c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f52d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [O-2]: 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.merge_operator: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_filter_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.sst_partitioner_factory: None 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_factory: SkipListFactory 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_factory: BlockBasedTable 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x38f03c0) cache_index_and_filter_blocks: 1 cache_index_and_filter_blocks_with_high_priority: 0 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 checksum: 4 no_block_cache: 0 block_cache: 0x24f52d0 block_cache_name: BinnedLRUCache block_cache_options: capacity : 536870912 num_shard_bits : 4 strict_capacity_limit : 0 high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: bloomfilter whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 initial_auto_readahead_size: 8192 num_file_reads_for_auto_readahead: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.write_buffer_size: 16777216 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number: 64 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression: LZ4 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression: Disabled 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.num_levels: 7 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_write_buffer_number_to_merge: 6 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.window_bits: -14 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.level: 32767 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.strategy: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.enabled: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_file_num_compaction_trigger: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_base: 67108864 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.target_file_size_multiplier: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_base: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier: 8.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.arena_block_size: 1048576 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.disable_auto_compactions: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_support: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.inplace_update_num_locks: 10000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.memtable_huge_page_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.bloom_locality: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.max_successive_merges: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.optimize_filters_for_hits: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.paranoid_file_checks: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.force_consistency_checks: 1 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.report_bg_io_stats: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.ttl: 2592000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.periodic_compaction_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_files: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.min_blob_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_size: 268435456 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compression_type: NoCompression 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.enable_blob_garbage_collection: false 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.blob_file_starting_level: 0 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:07.156+0000 7fac73829780 4 rocksdb: [db/column_family.cc:635] (skipping printing options) 2026-02-11T23:30:07.160+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:db/MANIFEST-000032 succeeded,manifest_file_number is 32, next_file_number is 34, last_sequence is 11, log_number is 5,prev_log_number is 0,max_column_family is 11,min_log_number_to_keep is 5 2026-02-11T23:30:07.160+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2026-02-11T23:30:07.160+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [m-0] (ID 1), log number is 5 2026-02-11T23:30:07.160+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [m-1] (ID 2), log number is 5 2026-02-11T23:30:07.160+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [m-2] (ID 3), log number is 5 2026-02-11T23:30:07.160+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [p-0] (ID 4), log number is 5 2026-02-11T23:30:07.160+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [p-1] (ID 5), log number is 5 2026-02-11T23:30:07.160+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [p-2] (ID 6), log number is 5 2026-02-11T23:30:07.160+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [O-0] (ID 7), log number is 5 2026-02-11T23:30:07.160+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [O-1] (ID 8), log number is 5 2026-02-11T23:30:07.160+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [O-2] (ID 9), log number is 5 2026-02-11T23:30:07.161+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [L] (ID 10), log number is 5 2026-02-11T23:30:07.161+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5581] Column family [P] (ID 11), log number is 5 2026-02-11T23:30:07.161+0000 7fac73829780 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c950e167-b477-40c0-82b4-f501f07d8a84 2026-02-11T23:30:07.161+0000 7fac73829780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852607161427, "job": 1, "event": "recovery_started", "wal_files": [31]} 2026-02-11T23:30:07.161+0000 7fac73829780 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2026-02-11T23:30:07.161+0000 7fac73829780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852607161913, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 1273, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12, "largest_seqno": 19, "table_properties": {"data_size": 130, "index_size": 27, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 92, "raw_average_key_size": 18, "raw_value_size": 70, "raw_average_value_size": 14, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": ".T:int64_array.b:bitwise_xor", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "LZ4", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1770852607, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c950e167-b477-40c0-82b4-f501f07d8a84", "db_session_id": "GFOOOOZ5B5960W5C3D63", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2026-02-11T23:30:07.161+0000 7fac73829780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852607162155, "cf_name": "p-0", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1594, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 14, "largest_seqno": 15, "table_properties": {"data_size": 469, "index_size": 39, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 72, "raw_average_key_size": 36, "raw_value_size": 571, "raw_average_value_size": 285, "num_data_blocks": 1, "num_entries": 2, "num_filter_entries": 2, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "p-0", "column_family_id": 4, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "LZ4", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1770852607, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c950e167-b477-40c0-82b4-f501f07d8a84", "db_session_id": "GFOOOOZ5B5960W5C3D63", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2026-02-11T23:30:07.161+0000 7fac73829780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852607162390, "cf_name": "O-2", "job": 1, "event": "table_file_creation", "file_number": 37, "file_size": 1274, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16, "largest_seqno": 16, "table_properties": {"data_size": 121, "index_size": 64, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 55, "raw_average_key_size": 55, "raw_value_size": 50, "raw_average_value_size": 50, "num_data_blocks": 1, "num_entries": 1, "num_filter_entries": 1, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "O-2", "column_family_id": 9, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "LZ4", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1770852607, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c950e167-b477-40c0-82b4-f501f07d8a84", "db_session_id": "GFOOOOZ5B5960W5C3D63", "orig_file_number": 37, "seqno_to_time_mapping": "N/A"}} 2026-02-11T23:30:07.162+0000 7fac73829780 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1770852607162423, "job": 1, "event": "recovery_finished"} 2026-02-11T23:30:07.162+0000 7fac73829780 4 rocksdb: [db/version_set.cc:5047] Creating manifest 39 2026-02-11T23:30:07.164+0000 7fac73829780 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x38b2000 2026-02-11T23:30:07.164+0000 7fac73829780 4 rocksdb: DB pointer 0x386e000 2026-02-11T23:30:07.164+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-11T23:30:07.164+0000 7fac5e182640 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2026-02-11T23:30:07.164+0000 7fac5e182640 4 rocksdb: [db/db_impl/db_impl.cc:1111] ** DB Stats ** Uptime(secs): 0.0 total, 0.0 interval Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 2/0 2.62 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 0.0 0.0 Sum 2/0 2.62 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.4 0.00 0.00 1 0.000 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f4d30#315501 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [default] ** ** Compaction Stats [m-0] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [m-0] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f4d30#315501 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [m-0] ** ** Compaction Stats [m-1] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [m-1] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f4d30#315501 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [m-1] ** ** Compaction Stats [m-2] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [m-2] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f4d30#315501 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [m-2] ** ** Compaction Stats [p-0] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 1/0 1.56 KB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.9 0.00 0.00 1 0.000 0 0 0.0 0.0 Sum 1/0 1.56 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.9 0.00 0.00 1 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 6.9 0.00 0.00 1 0.000 0 0 0.0 0.0 ** Compaction Stats [p-0] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 6.9 0.00 0.00 1 0.000 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f4d30#315501 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [p-0] ** ** Compaction Stats [p-1] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [p-1] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f4d30#315501 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [p-1] ** ** Compaction Stats [p-2] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [p-2] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f4d30#315501 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [p-2] ** ** Compaction Stats [O-0] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [O-0] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f52d0#315501 capacity: 512.00 MB usage: 0.25 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 2 last_secs: 2e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.14 KB,2.68221e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [O-0] ** ** Compaction Stats [O-1] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [O-1] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f52d0#315501 capacity: 512.00 MB usage: 0.25 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 2 last_secs: 2e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.14 KB,2.68221e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [O-1] ** ** Compaction Stats [O-2] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 1/0 1.24 KB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.5 0.00 0.00 1 0.000 0 0 0.0 0.0 Sum 1/0 1.24 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.5 0.00 0.00 1 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.5 0.00 0.00 1 0.000 0 0 0.0 0.0 ** Compaction Stats [O-2] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.5 0.00 0.00 1 0.000 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f52d0#315501 capacity: 512.00 MB usage: 0.25 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 2 last_secs: 2e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.14 KB,2.68221e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [O-2] ** ** Compaction Stats [L] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [L] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f4d30#315501 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [L] ** ** Compaction Stats [P] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 ** Compaction Stats [P] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 0.0 total, 0.0 interval Flush(GB): cumulative 0.000, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Block cache BinnedLRUCache@0x24f4d30#315501 capacity: 1.35 GB usage: 0.69 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 8 last_secs: 9e-06 secs_since: 0 Block cache entry stats(count,size,portion): FilterBlock(3,0.33 KB,2.31796e-05%) IndexBlock(3,0.36 KB,2.53872e-05%) Misc(1,0.00 KB,0%) ** File Read Latency Histogram By Level [P] ** 2026-02-11T23:30:07.164+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _upgrade_super from 4, latest 4 2026-02-11T23:30:07.164+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) _upgrade_super done 2026-02-11T23:30:07.164+0000 7fac73829780 2 osd.0 0 journal looks like ssd 2026-02-11T23:30:07.164+0000 7fac73829780 2 osd.0 0 boot 2026-02-11T23:30:07.165+0000 7fac73829780 20 osd.0 0 configured osd_max_object_name[space]_len looks ok 2026-02-11T23:30:07.165+0000 7fac73829780 10 osd.0 0 read_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e0 maps [] lci=[0,0] tlb=0) 2026-02-11T23:30:07.165+0000 7fac73829780 20 osd.0 0 get_map 0 - return initial 0x2631e00 2026-02-11T23:30:07.165+0000 7fac73829780 10 osd.0 0 init creating/touching snapmapper object 2026-02-11T23:30:07.165+0000 7fac73829780 10 osd.0 0 init creating/touching purged_snaps object 2026-02-11T23:30:07.165+0000 7fac73829780 10 open_all_classes 2026-02-11T23:30:07.165+0000 7fac73829780 10 open_all_classes found journal 2026-02-11T23:30:07.165+0000 7fac73829780 10 _get_class adding new class name journal 0x391d580 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class journal from /usr/lib64/rados-classes/libcls_journal.so 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_class journal status 3 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.create flags 3 0x7fac6297e400 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.get_order flags 1 0x7fac6297de20 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.get_splay_width flags 1 0x7fac6297de90 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.get_pool_id flags 1 0x7fac6297df00 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.get_minimum_set flags 1 0x7fac6297e0a0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.set_minimum_set flags 3 0x7fac6297e720 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.get_active_set flags 1 0x7fac6297e110 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.set_active_set flags 3 0x7fac6297e850 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.get_client flags 1 0x7fac629801e0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.client_register flags 3 0x7fac62980820 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.client_update_data flags 3 0x7fac62981650 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.client_update_state flags 3 0x7fac62981a80 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.client_unregister flags 3 0x7fac62985210 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.client_commit flags 3 0x7fac62981dc0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.client_list flags 1 0x7fac62980460 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.get_next_tag_tid flags 1 0x7fac6297e180 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.get_tag flags 1 0x7fac62983a20 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.tag_create flags 3 0x7fac62985430 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.tag_list flags 1 0x7fac629823e0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.guard_append flags 3 0x7fac6297e980 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method journal.append flags 3 0x7fac6297eab0 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class journal success 2026-02-11T23:30:07.165+0000 7fac73829780 10 open_all_classes found lock 2026-02-11T23:30:07.165+0000 7fac73829780 10 _get_class adding new class name lock 0x391d6c0 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class lock from /usr/lib64/rados-classes/libcls_lock.so 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_class lock status 3 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method lock.lock flags 11 0x7fac6296e5b0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method lock.unlock flags 11 0x7fac6296bc20 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method lock.break_lock flags 3 0x7fac6296bef0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method lock.get_info flags 1 0x7fac6296daf0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method lock.list_locks flags 1 0x7fac629692c0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method lock.assert_locked flags 9 0x7fac6296c1a0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method lock.set_cookie flags 11 0x7fac6296ca60 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class lock success 2026-02-11T23:30:07.165+0000 7fac73829780 10 open_all_classes found fifo 2026-02-11T23:30:07.165+0000 7fac73829780 10 _get_class adding new class name fifo 0x391d800 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class fifo from /usr/lib64/rados-classes/libcls_fifo.so 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_class fifo status 3 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method fifo.create_meta flags 3 0x7fac62939fd0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method fifo.get_meta flags 1 0x7fac6293c610 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method fifo.update_meta flags 3 0x7fac6293b730 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method fifo.init_part flags 3 0x7fac6293de00 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method fifo.push_part flags 3 0x7fac6293e1f0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method fifo.trim_part flags 3 0x7fac6293f160 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method fifo.part_list flags 1 0x7fac6293f770 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method fifo.get_part_info flags 1 0x7fac6294cb10 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class fifo success 2026-02-11T23:30:07.165+0000 7fac73829780 10 open_all_classes found hello 2026-02-11T23:30:07.165+0000 7fac73829780 10 _get_class adding new class name hello 0x391d940 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class hello from /usr/lib64/rados-classes/libcls_hello.so 2026-02-11T23:30:07.165+0000 7fac73829780 0 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/20.3.0-5256-gf2a98edf/rpm/el9/BUILD/ceph-20.3.0-5256-gf2a98edf/src/cls/hello/cls_hello.cc:316: loading cls_hello 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_class hello status 3 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method hello.say_hello flags 1 0x7fac72e0d0a0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method hello.record_hello flags 10 0x7fac72e0e1c0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method hello.write_return_data flags 2 0x7fac72e0d130 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method hello.writes_dont_return_data flags 2 0x7fac72e0d130 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method hello.write_too_much_return_data flags 2 0x7fac72e0d250 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method hello.replay flags 1 0x7fac72e0d370 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method hello.turn_it_to_11 flags 11 0x7fac72e0d760 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method hello.bad_reader flags 2 0x7fac72e0d3a0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method hello.bad_writer flags 1 0x7fac72e0d360 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class hello success 2026-02-11T23:30:07.165+0000 7fac73829780 10 open_all_classes found cmpomap 2026-02-11T23:30:07.165+0000 7fac73829780 10 _get_class adding new class name cmpomap 0x391da80 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class cmpomap from /usr/lib64/rados-classes/libcls_cmpomap.so 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_class cmpomap status 3 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method cmpomap.cmp_vals flags 1 0x7fac62922470 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method cmpomap.cmp_set_vals flags 3 0x7fac62922ee0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method cmpomap.cmp_rm_keys flags 3 0x7fac62924550 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class cmpomap success 2026-02-11T23:30:07.165+0000 7fac73829780 10 open_all_classes found refcount 2026-02-11T23:30:07.165+0000 7fac73829780 10 _get_class adding new class name refcount 0x391c040 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class refcount from /usr/lib64/rados-classes/libcls_refcount.so 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_class refcount status 3 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method refcount.get flags 3 0x7fac629089f0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method refcount.put flags 3 0x7fac62908e80 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method refcount.set flags 3 0x7fac62907160 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method refcount.read flags 1 0x7fac629095e0 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class refcount success 2026-02-11T23:30:07.165+0000 7fac73829780 10 open_all_classes found numops 2026-02-11T23:30:07.165+0000 7fac73829780 10 _get_class adding new class name numops 0x391c900 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class numops from /usr/lib64/rados-classes/libcls_numops.so 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_class numops status 3 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method numops.add flags 3 0x7fac735c6170 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method numops.mul flags 3 0x7fac735c6c20 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class numops success 2026-02-11T23:30:07.165+0000 7fac73829780 10 open_all_classes found rbd 2026-02-11T23:30:07.165+0000 7fac73829780 10 _get_class adding new class name rbd 0x391ca40 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class rbd from /usr/lib64/rados-classes/libcls_rbd.so 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_class rbd status 3 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.create flags 3 0x7fac628ab850 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_features flags 1 0x7fac628a9da0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.set_features flags 3 0x7fac628a9f70 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_size flags 1 0x7fac628ac5f0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.set_size flags 3 0x7fac628aa2b0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_snapcontext flags 1 0x7fac628b1570 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_object_prefix flags 1 0x7fac628ab300 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_data_pool flags 1 0x7fac628ab4a0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_snapshot_name flags 1 0x7fac628ad3c0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_snapshot_timestamp flags 1 0x7fac628ad610 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.snapshot_get flags 1 0x7fac628ad820 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.snapshot_add flags 3 0x7fac628b3e60 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.snapshot_remove flags 3 0x7fac628bcd40 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.snapshot_rename flags 3 0x7fac628b4e90 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.snapshot_trash_add flags 3 0x7fac628bee80 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_all_features flags 1 0x7fac628b5a10 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_parent flags 1 0x7fac628afa10 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.set_parent flags 3 0x7fac628a9b10 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.remove_parent flags 3 0x7fac628b3520 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.parent_get flags 1 0x7fac628b3560 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.parent_overlap_get flags 1 0x7fac628acf50 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.parent_attach flags 3 0x7fac628b5640 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.parent_detach flags 3 0x7fac628b3540 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.set_protection_status flags 3 0x7fac628b0950 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_protection_status flags 1 0x7fac628ac9b0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_stripe_unit_count flags 1 0x7fac628aa6e0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.set_stripe_unit_count flags 3 0x7fac628aa930 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_create_timestamp flags 1 0x7fac628aad00 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_access_timestamp flags 1 0x7fac628aaf00 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_modify_timestamp flags 1 0x7fac628ab100 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_flags flags 1 0x7fac628acc50 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.set_flags flags 3 0x7fac628b0c50 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.op_features_get flags 1 0x7fac628a87b0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.op_features_set flags 3 0x7fac628ae4d0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.metadata_list flags 1 0x7fac628baf00 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.metadata_set flags 3 0x7fac628bf2a0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.metadata_remove flags 3 0x7fac628b77d0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.metadata_get flags 1 0x7fac628bac90 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.snapshot_get_limit flags 1 0x7fac628bb6b0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.snapshot_set_limit flags 3 0x7fac628bb810 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.child_attach flags 3 0x7fac628bd510 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.child_detach flags 3 0x7fac628bd9e0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.children_list flags 1 0x7fac628bea20 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.migration_set flags 3 0x7fac628bbc30 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.migration_set_state flags 3 0x7fac628bbea0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.migration_get flags 1 0x7fac628bc1f0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.migration_remove flags 3 0x7fac628bc420 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.set_modify_timestamp flags 3 0x7fac628b6420 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.set_access_timestamp flags 3 0x7fac628b6330 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.add_child flags 3 0x7fac628a8b50 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.remove_child flags 3 0x7fac628a9020 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_children flags 1 0x7fac628b1230 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.get_id flags 1 0x7fac628b5ed0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.set_id flags 3 0x7fac628b6110 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.dir_get_id flags 1 0x7fac628b6510 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.dir_get_name flags 1 0x7fac628b6780 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.dir_list flags 1 0x7fac628b7970 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.dir_add_image flags 3 0x7fac628b6e60 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.dir_remove_image flags 3 0x7fac628b7670 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.dir_rename_image flags 3 0x7fac628b7460 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.dir_state_assert flags 1 0x7fac628b8470 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.dir_state_set flags 3 0x7fac628b8720 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.object_map_load flags 1 0x7fac628b8990 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.object_map_save flags 3 0x7fac628b8aa0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.object_map_resize flags 3 0x7fac628b8c80 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.object_map_update flags 3 0x7fac628ba1a0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.object_map_snap_add flags 3 0x7fac628b9610 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.object_map_snap_remove flags 3 0x7fac628b9a80 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.snap_list flags 1 0x7fac628bc540 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.snap_add flags 3 0x7fac628bc880 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.snap_remove flags 3 0x7fac628bfdd0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.snap_rename flags 3 0x7fac628c02d0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_uuid_get flags 1 0x7fac628c0c60 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_uuid_set flags 3 0x7fac628c8aa0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_mode_get flags 1 0x7fac628c0d40 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_mode_set flags 3 0x7fac628c57d0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_remote_namespace_get flags 1 0x7fac628c1010 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_remote_namespace_set flags 3 0x7fac628c0da0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_peer_ping flags 3 0x7fac628ca160 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_peer_list flags 1 0x7fac628c5b50 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_peer_add flags 3 0x7fac628c60b0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_peer_remove flags 3 0x7fac628c0ad0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_peer_set_client flags 3 0x7fac628c1330 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_peer_set_cluster flags 3 0x7fac628c5d00 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_peer_set_direction flags 3 0x7fac628c1620 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_list flags 1 0x7fac628c1890 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_get_image_id flags 1 0x7fac628c10f0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_get flags 1 0x7fac628c2320 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_set flags 3 0x7fac628c2480 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_remove flags 3 0x7fac628c7160 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_status_set flags 11 0x7fac628caee0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_status_remove flags 3 0x7fac628c7560 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_status_get flags 1 0x7fac628c2b00 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_status_list flags 1 0x7fac628c2cb0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_status_get_summary flags 1 0x7fac628c8d00 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_status_remove_down flags 3 0x7fac628cc1a0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_instance_get flags 1 0x7fac628c7940 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_instance_list flags 1 0x7fac628c7c90 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_instances_list flags 1 0x7fac628d0990 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_instances_add flags 11 0x7fac628cc1c0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_instances_remove flags 3 0x7fac628cc3e0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_map_list flags 1 0x7fac628cf7a0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_map_update flags 2 0x7fac628cc730 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_map_remove flags 2 0x7fac628cc570 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_snapshot_unlink_peer flags 3 0x7fac628d57f0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.mirror_image_snapshot_set_copy_progress flags 3 0x7fac628d5bc0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_dir_list flags 1 0x7fac628cd7b0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_dir_add flags 3 0x7fac628ce530 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_dir_remove flags 3 0x7fac628cce60 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_dir_rename flags 3 0x7fac628cd410 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_image_remove flags 3 0x7fac628ce700 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_image_list flags 1 0x7fac628d43b0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_image_set flags 3 0x7fac628d5ef0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.image_group_add flags 3 0x7fac628ce8b0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.image_group_remove flags 3 0x7fac628cebe0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.image_group_get flags 1 0x7fac628ceee0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_snap_set flags 3 0x7fac628d2a00 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_snap_remove flags 3 0x7fac628d36a0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_snap_get_by_id flags 1 0x7fac628d2350 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_snap_list flags 1 0x7fac628d1f60 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.group_snap_list_order flags 1 0x7fac628d3950 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.trash_add flags 3 0x7fac628cf360 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.trash_remove flags 3 0x7fac628cf0c0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.trash_list flags 1 0x7fac628db480 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.trash_get flags 1 0x7fac628d6220 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.trash_state_set flags 3 0x7fac628d6470 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.namespace_add flags 3 0x7fac628d6860 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.namespace_remove flags 3 0x7fac628d6a70 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.namespace_list flags 1 0x7fac628d6c30 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.copyup flags 3 0x7fac628b5a60 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.sparse_copyup flags 3 0x7fac628b5ad0 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.assert_snapc_seq flags 3 0x7fac628bc440 2026-02-11T23:30:07.165+0000 7fac73829780 10 register_cxx_method rbd.sparsify flags 3 0x7fac628d7480 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class rbd success 2026-02-11T23:30:07.165+0000 7fac73829780 10 open_all_classes found otp 2026-02-11T23:30:07.165+0000 7fac73829780 10 _get_class adding new class name otp 0x391cb80 2026-02-11T23:30:07.165+0000 7fac73829780 10 _load_class otp from /usr/lib64/rados-classes/libcls_otp.so 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class otp status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method otp.otp_set flags 3 0x7fac6288b070 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method otp.otp_get flags 1 0x7fac6288cd60 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method otp.otp_check flags 3 0x7fac6288daf0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method otp.otp_get_result flags 1 0x7fac6288e2b0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method otp.otp_remove flags 3 0x7fac6288c190 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method otp.get_current_time flags 1 0x7fac6288a290 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class otp success 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found sem_set 2026-02-11T23:30:07.166+0000 7fac73829780 10 _get_class adding new class name sem_set 0x391ccc0 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class sem_set from /usr/lib64/rados-classes/libcls_sem_set.so 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class sem_set status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method sem_set.increment flags 3 0x7fac62860800 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method sem_set.decrement flags 3 0x7fac62860f80 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method sem_set.reset flags 3 0x7fac6285c9b0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method sem_set.list flags 1 0x7fac6285d640 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class sem_set success 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found sdk 2026-02-11T23:30:07.166+0000 7fac73829780 0 _get_class not permitted to load sdk 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found lua 2026-02-11T23:30:07.166+0000 7fac73829780 0 _get_class not permitted to load lua 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found timeindex 2026-02-11T23:30:07.166+0000 7fac73829780 10 _get_class adding new class name timeindex 0x391ce00 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class timeindex from /usr/lib64/rados-classes/libcls_timeindex.so 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class timeindex status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method timeindex.add flags 3 0x7fac6284e1a0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method timeindex.list flags 1 0x7fac6284e710 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method timeindex.trim flags 3 0x7fac6284fa80 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class timeindex success 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found log 2026-02-11T23:30:07.166+0000 7fac73829780 10 _get_class adding new class name log 0x391d080 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class log from /usr/lib64/rados-classes/libcls_log.so 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class log status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method log.add flags 3 0x7fac62842d90 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method log.list flags 1 0x7fac628416f0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method log.trim flags 3 0x7fac62840900 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method log.info flags 1 0x7fac62843a30 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class log success 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found rgw 2026-02-11T23:30:07.166+0000 7fac73829780 10 _get_class adding new class name rgw 0x391d1c0 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class rgw from /usr/lib64/rados-classes/libcls_rgw.so 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class rgw status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_init_index flags 3 0x7fac627d35f0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_init_index2 flags 3 0x7fac627d35f0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_set_tag_timeout flags 3 0x7fac627d48d0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_list flags 1 0x7fac627cf5f0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_check_index flags 1 0x7fac627e4b80 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_rebuild_index flags 3 0x7fac627e48d0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_update_stats flags 3 0x7fac627da630 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_prepare_op flags 3 0x7fac627d4b40 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_complete_op flags 3 0x7fac627d5860 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_link_olh flags 3 0x7fac627d79e0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_unlink_instance flags 3 0x7fac627d9310 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_read_olh_log flags 1 0x7fac627dae90 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_trim_olh_log flags 3 0x7fac627dc400 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bucket_clear_olh flags 3 0x7fac627df790 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.obj_remove flags 3 0x7fac627dcb80 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.obj_store_pg_ver flags 2 0x7fac627dd520 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.obj_check_attrs_prefix flags 1 0x7fac627dd820 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.obj_check_mtime flags 1 0x7fac627dddb0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bi_get flags 1 0x7fac627de190 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bi_put flags 3 0x7fac627de700 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bi_put_entries flags 3 0x7fac627de9d0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bi_list flags 1 0x7fac627e7360 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.reshard_log_trim flags 3 0x7fac627e4e20 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bi_log_list flags 1 0x7fac627e9060 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bi_log_trim flags 3 0x7fac627ea1f0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.dir_suggest_changes flags 3 0x7fac627dffa0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bi_log_resync flags 3 0x7fac627e5240 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.bi_log_stop flags 3 0x7fac627e5660 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.user_usage_log_add flags 3 0x7fac627ea7c0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.user_usage_log_read flags 1 0x7fac627ee350 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.user_usage_log_trim flags 3 0x7fac627eea90 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.usage_log_clear flags 2 0x7fac627ed2f0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.gc_set_entry flags 3 0x7fac627eedf0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.gc_defer_entry flags 3 0x7fac627ef090 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.gc_list flags 1 0x7fac627ef3c0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.gc_remove flags 3 0x7fac627f24f0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.lc_get_entry flags 1 0x7fac627f0380 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.lc_set_entry flags 3 0x7fac627f0820 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.lc_rm_entry flags 3 0x7fac627f0c20 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.lc_get_next_entry flags 1 0x7fac627f0ed0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.lc_put_head flags 3 0x7fac627f2200 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.lc_get_head flags 1 0x7fac627ed880 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.lc_list_entries flags 1 0x7fac627f1450 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.mp_upload_part_info_update flags 3 0x7fac627f6b80 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.reshard_add flags 3 0x7fac627f91d0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.reshard_list flags 1 0x7fac627f9640 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.reshard_get flags 1 0x7fac627fa120 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.reshard_remove flags 3 0x7fac627fa680 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.set_bucket_resharding flags 3 0x7fac627fab80 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.clear_bucket_resharding flags 3 0x7fac627fae00 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.guard_bucket_resharding flags 1 0x7fac627fb060 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw.get_bucket_resharding flags 1 0x7fac627fb320 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class rgw success 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found 2pc_queue 2026-02-11T23:30:07.166+0000 7fac73829780 10 _get_class adding new class name 2pc_queue 0x391d300 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class 2pc_queue from /usr/lib64/rados-classes/libcls_2pc_queue.so 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class 2pc_queue status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method 2pc_queue.2pc_queue_init flags 3 0x7fac6277fb50 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method 2pc_queue.2pc_queue_get_capacity flags 1 0x7fac6277ef50 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method 2pc_queue.2pc_queue_get_topic_stats flags 1 0x7fac62780d70 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method 2pc_queue.2pc_queue_reserve flags 3 0x7fac62781c90 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method 2pc_queue.2pc_queue_commit flags 3 0x7fac627826e0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method 2pc_queue.2pc_queue_abort flags 3 0x7fac627842a0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method 2pc_queue.2pc_queue_list_reservations flags 1 0x7fac627848f0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method 2pc_queue.2pc_queue_list_entries flags 1 0x7fac62780270 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method 2pc_queue.2pc_queue_remove_entries flags 3 0x7fac62781010 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method 2pc_queue.2pc_queue_expire_reservations flags 3 0x7fac62785170 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class 2pc_queue success 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found cas 2026-02-11T23:30:07.166+0000 7fac73829780 10 _get_class adding new class name cas 0x391d440 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class cas from /usr/lib64/rados-classes/libcls_cas.so 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class cas status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method cas.chunk_create_or_get_ref flags 3 0x7fac6276c2c0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method cas.chunk_get_ref flags 3 0x7fac6276c8a0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method cas.chunk_put_ref flags 3 0x7fac6276cbb0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method cas.references_chunk flags 1 0x7fac62768c40 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class cas success 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found user 2026-02-11T23:30:07.166+0000 7fac73829780 10 _get_class adding new class name user 0x391c540 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class user from /usr/lib64/rados-classes/libcls_user.so 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class user status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method user.set_buckets_info flags 3 0x7fac6274fc20 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method user.complete_stats_sync flags 3 0x7fac62751600 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method user.remove_bucket flags 3 0x7fac62751a10 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method user.list_buckets flags 1 0x7fac62752510 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method user.get_header flags 1 0x7fac627539d0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method user.reset_user_stats flags 3 0x7fac62753ca0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method user.reset_user_stats2 flags 3 0x7fac627547e0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method user.account_resource_add flags 3 0x7fac62755600 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method user.account_resource_get flags 1 0x7fac62755d60 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method user.account_resource_rm flags 3 0x7fac627562b0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method user.account_resource_list flags 1 0x7fac627566b0 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class user success 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found version 2026-02-11T23:30:07.166+0000 7fac73829780 10 _get_class adding new class name version 0x391c400 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class version from /usr/lib64/rados-classes/libcls_version.so 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class version status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method version.set flags 3 0x7fac6273e8e0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method version.inc flags 3 0x7fac6273ec20 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method version.inc_conds flags 3 0x7fac6273ec20 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method version.read flags 1 0x7fac6273f320 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method version.check_conds flags 1 0x7fac6273efa0 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class version success 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found rgw_gc 2026-02-11T23:30:07.166+0000 7fac73829780 10 _get_class adding new class name rgw_gc 0x391c2c0 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class rgw_gc from /usr/lib64/rados-classes/libcls_rgw_gc.so 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class rgw_gc status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw_gc.rgw_gc_queue_init flags 3 0x7fac627205c0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw_gc.rgw_gc_queue_enqueue flags 3 0x7fac62720c70 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw_gc.rgw_gc_queue_list_entries flags 1 0x7fac62721cc0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw_gc.rgw_gc_queue_remove_entries flags 3 0x7fac62723910 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method rgw_gc.rgw_gc_queue_update_entry flags 3 0x7fac62724f10 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class rgw_gc success 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found queue 2026-02-11T23:30:07.166+0000 7fac73829780 10 _get_class adding new class name queue 0x391c180 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class queue from /usr/lib64/rados-classes/libcls_queue.so 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class queue status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method queue.queue_init flags 3 0x7fac626ffc70 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method queue.queue_get_capacity flags 1 0x7fac62700200 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method queue.queue_enqueue flags 3 0x7fac62702fd0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method queue.queue_list_entries flags 1 0x7fac627003a0 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method queue.queue_remove_entries flags 3 0x7fac627028e0 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class queue success 2026-02-11T23:30:07.166+0000 7fac73829780 10 open_all_classes found cephfs 2026-02-11T23:30:07.166+0000 7fac73829780 10 _get_class adding new class name cephfs 0x254ab80 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class cephfs from /usr/lib64/rados-classes/libcls_cephfs.so 2026-02-11T23:30:07.166+0000 7fac73829780 0 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/20.3.0-5256-gf2a98edf/rpm/el9/BUILD/ceph-20.3.0-5256-gf2a98edf/src/cls/cephfs/cls_cephfs.cc:202: loading cephfs 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_class cephfs status 3 2026-02-11T23:30:07.166+0000 7fac73829780 10 register_cxx_method cephfs.accumulate_inode_metadata flags 3 0x7fac626eefc0 2026-02-11T23:30:07.166+0000 7fac73829780 10 _load_class cephfs success 2026-02-11T23:30:07.166+0000 7fac73829780 0 osd.0 0 crush map has features 288232575208783872, adjusting msgr requires for clients 2026-02-11T23:30:07.166+0000 7fac73829780 0 osd.0 0 crush map has features 288232575208783872 was 8705, adjusting msgr requires for mons 2026-02-11T23:30:07.166+0000 7fac73829780 0 osd.0 0 crush map has features 288232575208783872, adjusting msgr requires for osds 2026-02-11T23:30:07.166+0000 7fac73829780 0 osd.0 0 check_osdmap_features enabling on-disk ERASURE CODES compat feature 2026-02-11T23:30:07.166+0000 7fac73829780 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e0 maps [] lci=[0,0] tlb=0) 2026-02-11T23:30:07.166+0000 7fac73829780 10 osd.0 0 clear_temp_objects 2026-02-11T23:30:07.166+0000 7fac73829780 0 osd.0 0 load_pgs 2026-02-11T23:30:07.166+0000 7fac73829780 20 osd.0 0 load_pgs pg_num_history pg_num_history(e0 pg_nums {} deleted_pools ) 2026-02-11T23:30:07.166+0000 7fac73829780 10 osd.0 0 load_pgs ignoring unrecognized meta 2026-02-11T23:30:07.166+0000 7fac73829780 0 osd.0 0 load_pgs opened 0 pgs 2026-02-11T23:30:07.166+0000 7fac73829780 2 osd.0 0 superblock: I am osd.0 2026-02-11T23:30:07.166+0000 7fac73829780 0 osd.0 0 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:07.166+0000 7fac73829780 10 monclient: init 2026-02-11T23:30:07.166+0000 7fac73829780 1 Processor -- start 2026-02-11T23:30:07.167+0000 7fac73829780 20 log_channel(cluster) update_config log_to_monitors true log_to_syslog false log_channels log_prios info 2026-02-11T23:30:07.167+0000 7fac73829780 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201) 2026-02-11T23:30:07.167+0000 7fac73829780 -1 osd.0 0 log_to_monitors true 2026-02-11T23:30:07.167+0000 7fac73829780 1 Processor -- start 2026-02-11T23:30:07.167+0000 7fac73829780 1 Processor -- start 2026-02-11T23:30:07.167+0000 7fac73829780 1 Processor -- start 2026-02-11T23:30:07.167+0000 7fac73829780 1 Processor -- start 2026-02-11T23:30:07.167+0000 7fac73829780 1 Processor -- start 2026-02-11T23:30:07.167+0000 7fac73829780 1 Processor -- start 2026-02-11T23:30:07.167+0000 7fac55971640 10 osd.0 0 agent_entry start 2026-02-11T23:30:07.167+0000 7fac55971640 20 osd.0 0 agent_entry empty queue 2026-02-11T23:30:07.167+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:07.167+0000 7fac5496f640 20 osd.0 op_wq(1) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac73829780 10 monclient: _reopen_session rank -1 2026-02-11T23:30:07.168+0000 7fac4d961640 20 osd.0 op_wq(7) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac73829780 10 monclient: _add_conns ranks=[0] 2026-02-11T23:30:07.168+0000 7fac5396d640 20 osd.0 op_wq(3) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac5416e640 20 osd.0 op_wq(2) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac5216a640 20 osd.0 op_wq(6) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac51969640 20 osd.0 op_wq(7) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac73829780 1 --2- [v2:0.0.0.0:6800/460666604,v1:0.0.0.0:6801/460666604] >> v2:127.0.0.1:7180/0 conn(0x396bc00 0x38a5080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:07.168+0000 7fac50967640 20 osd.0 op_wq(1) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac73829780 10 monclient(hunting): picked mon.noname-a con 0x396bc00 addr v2:127.0.0.1:7180/0 2026-02-11T23:30:07.168+0000 7fac5296b640 20 osd.0 op_wq(5) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac73829780 10 monclient(hunting): start opening mon connection 2026-02-11T23:30:07.168+0000 7fac5316c640 20 osd.0 op_wq(4) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac4f965640 20 osd.0 op_wq(3) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac4e963640 20 osd.0 op_wq(5) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac73829780 1 -- [v2:0.0.0.0:6800/460666604,v1:0.0.0.0:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_getmap magic: 0 -- 0x254a780 con 0x396bc00 2026-02-11T23:30:07.168+0000 7fac4e162640 20 osd.0 op_wq(6) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac73829780 10 monclient(hunting): _renew_subs 2026-02-11T23:30:07.168+0000 7fac50166640 20 osd.0 op_wq(2) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac4f164640 20 osd.0 op_wq(4) _process empty q, waiting 2026-02-11T23:30:07.168+0000 7fac4d160640 5 osd.0 0 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x256/0x1000, compress 0x0/0x0/0x0, omap 0x63a, meta 0x1a4f9c6), peers [] op hist []) 2026-02-11T23:30:07.168+0000 7fac4d160640 20 osd.0 0 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:07.168+0000 7fac723fe640 1 --2- [v2:0.0.0.0:6800/460666604,v1:0.0.0.0:6801/460666604] >> v2:127.0.0.1:7180/0 conn(0x396bc00 0x38a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:07.168+0000 7fac723fe640 1 --2- [v2:0.0.0.0:6800/460666604,v1:0.0.0.0:6801/460666604] >> v2:127.0.0.1:7180/0 conn(0x396bc00 0x38a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:127.0.0.1:7180/0 says I am v2:127.0.0.1:42330/0 (socket says 127.0.0.1:42330) 2026-02-11T23:30:07.168+0000 7fac723fe640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] learned_addr learned my addr [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] (peer_addr_for_me v2:127.0.0.1:0/0) 2026-02-11T23:30:07.168+0000 7fac723fe640 10 osd.0 0 new session (outgoing) 0x38e28c0 con=0x2519400 addr=v2:127.0.0.1:6800/460666604 2026-02-11T23:30:07.168+0000 7fac723fe640 10 monclient(hunting): get_auth_request con 0x396bc00 auth_method 0 2026-02-11T23:30:07.168+0000 7fac723fe640 10 monclient(hunting): get_auth_request method 1 preferred_modes [1] 2026-02-11T23:30:07.168+0000 7fac723fe640 10 monclient(hunting): _init_auth method 1 2026-02-11T23:30:07.168+0000 7fac723fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-11T23:30:07.168+0000 7fac723fe640 10 monclient(hunting): handle_auth_done global_id 4111 payload 0 2026-02-11T23:30:07.168+0000 7fac723fe640 10 monclient: _finish_hunting 0 2026-02-11T23:30:07.168+0000 7fac723fe640 1 monclient: found mon.noname-a 2026-02-11T23:30:07.168+0000 7fac723fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:07.168+0000 7fac723fe640 10 monclient: _send_mon_message to mon.noname-a at v2:127.0.0.1:7180/0 2026-02-11T23:30:07.168+0000 7fac723fe640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x38511e0 con 0x396bc00 2026-02-11T23:30:07.168+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> v2:127.0.0.1:7180/0 conn(0x396bc00 0x38a5080 crc :-1 s=READY pgs=15 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:07.168+0000 7fac626e8640 10 osd.0 0 ms_handle_connect con 0x396bc00 2026-02-11T23:30:07.168+0000 7fac626e8640 10 osd.0 0 ms_handle_connect on mon 2026-02-11T23:30:07.168+0000 7fac626e8640 10 osd.0 0 requeue_pg_temp 0 + 0 -> 0 2026-02-11T23:30:07.168+0000 7fac626e8640 20 osd.0 0 _send_ready_to_merge ready_to_merge_source {} not_ready_to_merge_source ready_to_merge_target {} not_ready_to_merge_target {} sent_ready_to_merge_source 2026-02-11T23:30:07.168+0000 7fac626e8640 20 osd.0 0 send_pg_created 2026-02-11T23:30:07.168+0000 7fac626e8640 10 osd.0 0 requeue_failures 0 + 0 -> 0 2026-02-11T23:30:07.168+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 1 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x25d8420 con 0x396bc00 2026-02-11T23:30:07.168+0000 7fac626e8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:07.168+0000 7fac626e8640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-11T23:30:07.168+0000 7fac626e8640 10 monclient: dump: epoch 1 fsid c7e182d9-676d-4eeb-80b1-915a88d421e5 last_changed 2026-02-11T23:30:05.661167+0000 created 2026-02-11T23:30:05.661167+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:07.168+0000 7fac626e8640 10 monclient: _finish_auth 0 2026-02-11T23:30:07.168+0000 7fac626e8640 10 monclient: _check_auth_tickets 2026-02-11T23:30:07.168+0000 7fac626e8640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:37.168816+0000) 2026-02-11T23:30:07.168+0000 7fac73829780 5 monclient: authenticate success, global_id 4111 2026-02-11T23:30:07.168+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 2 ==== config(0 keys) ==== 4+0+0 (crc 0 0 0) 0x3919b00 con 0x396bc00 2026-02-11T23:30:07.168+0000 7fac626e8640 10 monclient: handle_config config(0 keys) 2026-02-11T23:30:07.168+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 3 ==== mon_map magic: 0 ==== 178+0+0 (crc 0 0 0) 0x25d8b00 con 0x396bc00 2026-02-11T23:30:07.168+0000 7fac626e8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-11T23:30:07.168+0000 7fac626e8640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-11T23:30:07.168+0000 7fac626e8640 10 monclient: dump: epoch 1 fsid c7e182d9-676d-4eeb-80b1-915a88d421e5 last_changed 2026-02-11T23:30:05.661167+0000 created 2026-02-11T23:30:05.661167+0000 min_mon_release 20 (tentacle) election_strategy: 1 0: v2:127.0.0.1:7180/0 mon.a 2026-02-11T23:30:07.168+0000 7fac6f3f8640 4 set_mon_vals no callback set 2026-02-11T23:30:07.168+0000 7fac73829780 1 bluestore(td/osd-backfill-space/0) collect_metadata devices span numa nodes 0 2026-02-11T23:30:07.168+0000 7fac73829780 10 osd.0 0 mon_cmd_maybe_osd_create cmd: {"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} 2026-02-11T23:30:07.168+0000 7fac73829780 10 start_mon_command cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}] 2026-02-11T23:30:07.168+0000 7fac73829780 10 monclient: _send_command 1 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}] 2026-02-11T23:30:07.168+0000 7fac73829780 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:07.168+0000 7fac73829780 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_command({"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} v 0) -- 0x3919380 con 0x396bc00 2026-02-11T23:30:07.219+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 4 ==== mon_command_ack([{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]=0 set osd(s) 0 to class 'ssd' v3) ==== 131+0+0 (crc 0 0 0) 0x38511e0 con 0x396bc00 2026-02-11T23:30:07.219+0000 7fac626e8640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}] 2026-02-11T23:30:07.219+0000 7fac626e8640 10 monclient: _finish_command 1 = system:0 set osd(s) 0 to class 'ssd' 2026-02-11T23:30:07.219+0000 7fac73829780 10 osd.0 0 update_crush_location crush location is "host=trial114", "root=default" 2026-02-11T23:30:07.219+0000 7fac73829780 10 osd.0 0 mon_cmd_maybe_osd_create cmd: {"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial114", "root=default"]} 2026-02-11T23:30:07.219+0000 7fac73829780 10 start_mon_command cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial114", "root=default"]}] 2026-02-11T23:30:07.219+0000 7fac73829780 10 monclient: _send_command 2 [{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial114", "root=default"]}] 2026-02-11T23:30:07.219+0000 7fac73829780 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:07.219+0000 7fac73829780 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_command({"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial114", "root=default"]} v 0) -- 0x3919200 con 0x396bc00 2026-02-11T23:30:07.320+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 5 ==== mon_command_ack([{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial114", "root=default"]}]=0 create-or-move updating item name 'osd.0' weight 0.0977 at location {host=trial114,root=default} to crush map v4) ==== 250+0+0 (crc 0 0 0) 0x3850ea0 con 0x396bc00 2026-02-11T23:30:07.320+0000 7fac626e8640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0977, "args": ["host=trial114", "root=default"]}] 2026-02-11T23:30:07.320+0000 7fac626e8640 10 monclient: _finish_command 2 = system:0 create-or-move updating item name 'osd.0' weight 0.0977 at location {host=trial114,root=default} to crush map 2026-02-11T23:30:07.320+0000 7fac73829780 10 osd.0 0 ensuring pgs have consumed prior maps 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0 0 consume_map version 0 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0 0 await_reserved_maps epoch:0 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0 0 await_reserved_maps done 0 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0 0 consume_map 0 -- publish done 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.320+0000 7fac73829780 20 osd.0 0 prune_pg_created 2026-02-11T23:30:07.320+0000 7fac73829780 10 osd.0:0.consume_map 0 (was 0) 2026-02-11T23:30:07.320+0000 7fac73829780 10 osd.0:1.consume_map 0 (was 0) 2026-02-11T23:30:07.320+0000 7fac73829780 10 osd.0:2.consume_map 0 (was 0) 2026-02-11T23:30:07.320+0000 7fac73829780 10 osd.0:3.consume_map 0 (was 0) 2026-02-11T23:30:07.320+0000 7fac73829780 10 osd.0:4.consume_map 0 (was 0) 2026-02-11T23:30:07.320+0000 7fac73829780 10 osd.0:5.consume_map 0 (was 0) 2026-02-11T23:30:07.320+0000 7fac73829780 10 osd.0:6.consume_map 0 (was 0) 2026-02-11T23:30:07.320+0000 7fac73829780 10 osd.0:7.consume_map 0 (was 0) 2026-02-11T23:30:07.320+0000 7fac73829780 10 osd.0 0 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:07.320+0000 7fac73829780 0 osd.0 0 done with init, starting boot process 2026-02-11T23:30:07.320+0000 7fac73829780 10 monclient: _renew_subs 2026-02-11T23:30:07.320+0000 7fac73829780 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:07.320+0000 7fac73829780 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({mgrmap=0+,osd_pg_creates=0+}) -- 0x3850820 con 0x396bc00 2026-02-11T23:30:07.320+0000 7fac73829780 1 osd.0 0 start_boot 2026-02-11T23:30:07.320+0000 7fac73829780 10 osd.0 0 start_boot - have maps [] 2026-02-11T23:30:07.320+0000 7fac73829780 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:07.320+0000 7fac73829780 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_get_version(what=osdmap handle=1) -- 0x2580b00 con 0x396bc00 2026-02-11T23:30:07.320+0000 7fac73829780 1 osd.0 0 maybe_override_options_for_qos osd_max_backfills set to 10 2026-02-11T23:30:07.320+0000 7fac73829780 1 osd.0 0 maybe_override_options_for_qos osd_recovery_max_active set to 0 2026-02-11T23:30:07.320+0000 7fac73829780 1 osd.0 0 maybe_override_options_for_qos osd_recovery_max_active_hdd set to 3 2026-02-11T23:30:07.320+0000 7fac73829780 1 osd.0 0 maybe_override_options_for_qos osd_recovery_max_active_ssd set to 10 2026-02-11T23:30:07.320+0000 7fac73829780 0 OSDBenchTest: OSD Bench Test Params: count: 12288000 Bytes block size: 4 KiB number of objects: 100 object size: 4 MiB 2026-02-11T23:30:07.320+0000 7fac73829780 0 OSDBenchTest: run_test: running osd bench with count: 12288000 Bytes bsize: 4 KiB onum: 100 osize: 4 MiB 2026-02-11T23:30:07.321+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 6 ==== mgrmap(e 1) ==== 939+0+0 (crc 0 0 0) 0x3b1c000 con 0x396bc00 2026-02-11T23:30:07.321+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 7 ==== mon_get_version_reply(handle=1 version=4) ==== 24+0+0 (crc 0 0 0) 0x2580b00 con 0x396bc00 2026-02-11T23:30:07.321+0000 7fac626e8640 10 monclient: handle_get_version_reply finishing 1 version 4 2026-02-11T23:30:07.603+0000 7fac73829780 0 OSDBenchTest: prefill_objects: Prefill took 0.28 secs. 2026-02-11T23:30:07.647+0000 7fac73829780 0 OSDBenchTest: perform_write_test: Test took 0.04 secs. 2026-02-11T23:30:07.650+0000 7fac73829780 0 OSDBenchTest: cleanup: Clean-up done. 2026-02-11T23:30:07.650+0000 7fac73829780 0 OSDBenchTest: run_test: osd bench result - bandwidth: 264 MiB/s iops: 67517.90 elapsed_sec: 0.04 2026-02-11T23:30:07.650+0000 7fac73829780 10 osd.0 0 mon_cmd_set_config Set osd_mclock_max_capacity_iops_ssd = 67517.896294 2026-02-11T23:30:07.650+0000 7fac73829780 10 start_mon_command cmd=[{"prefix": "config set", "who": "osd.0", "name": "osd_mclock_max_capacity_iops_ssd", "value": "67517.896294"}] 2026-02-11T23:30:07.650+0000 7fac73829780 10 monclient: _send_command 3 [{"prefix": "config set", "who": "osd.0", "name": "osd_mclock_max_capacity_iops_ssd", "value": "67517.896294"}] 2026-02-11T23:30:07.650+0000 7fac73829780 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:07.650+0000 7fac73829780 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_command([{prefix=config set, name=osd_mclock_max_capacity_iops_ssd}] v 0) -- 0x3919980 con 0x396bc00 2026-02-11T23:30:07.650+0000 7fac6ebf7640 10 osd.0 0 _preboot _preboot mon has osdmaps 1..4 2026-02-11T23:30:07.650+0000 7fac6ebf7640 5 osd.0 0 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x256/0x1000, compress 0x0/0x0/0x0, omap 0x63a, meta 0x1a4f9c6), peers [] op hist []) 2026-02-11T23:30:07.650+0000 7fac6ebf7640 20 osd.0 0 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:07.650+0000 7fac6ebf7640 -1 osd.0 0 waiting for initial osdmap 2026-02-11T23:30:07.650+0000 7fac6ebf7640 10 monclient: _renew_subs 2026-02-11T23:30:07.650+0000 7fac6ebf7640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:07.650+0000 7fac6ebf7640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=1}) -- 0x38504e0 con 0x396bc00 2026-02-11T23:30:07.650+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 8 ==== osd_map(1..4 src has 1..4) ==== 2751+0+0 (crc 0 0 0) 0x38e36c0 con 0x396bc00 2026-02-11T23:30:07.650+0000 7fac626e8640 20 osd.0 0 OSD::ms_dispatch: osd_map(1..4 src has 1..4) 2026-02-11T23:30:07.650+0000 7fac626e8640 20 osd.0 0 _dispatch 0x38e36c0 osd_map(1..4 src has 1..4) 2026-02-11T23:30:07.650+0000 7fac626e8640 3 osd.0 0 handle_osd_map epochs [1,4], i have 0, src has [1,4] 2026-02-11T23:30:07.650+0000 7fac626e8640 10 osd.0 0 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:07.650+0000 7fac626e8640 10 osd.0 0 superblock cluster_osdmap_trim_lower_bound new epoch is: 1 2026-02-11T23:30:07.650+0000 7fac626e8640 10 osd.0 0 handle_osd_map got inc map for epoch 1 2026-02-11T23:30:07.650+0000 7fac626e8640 20 osd.0 0 got_full_map 1, nothing requested 2026-02-11T23:30:07.650+0000 7fac626e8640 10 osd.0 0 handle_osd_map got inc map for epoch 2 2026-02-11T23:30:07.650+0000 7fac626e8640 20 osd.0 0 got_full_map 2, nothing requested 2026-02-11T23:30:07.650+0000 7fac626e8640 10 osd.0 0 handle_osd_map got inc map for epoch 3 2026-02-11T23:30:07.650+0000 7fac626e8640 20 osd.0 0 got_full_map 3, nothing requested 2026-02-11T23:30:07.650+0000 7fac626e8640 10 osd.0 0 handle_osd_map got inc map for epoch 4 2026-02-11T23:30:07.650+0000 7fac626e8640 20 osd.0 0 got_full_map 4, nothing requested 2026-02-11T23:30:07.650+0000 7fac626e8640 10 osd.0 0 track_pools_and_pg_num_changes no maps stored, this is probably the first start of this osd 2026-02-11T23:30:07.650+0000 7fac626e8640 20 osd.0 0 handle_osd_map pg_num_history pg_num_history(e4 pg_nums {} deleted_pools ) 2026-02-11T23:30:07.650+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {1={},2={},3={},4={}} 2026-02-11T23:30:07.650+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:07.650+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e4 maps [1~4] lci=[0,0] tlb=1) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0 0 _committed_osd_maps 1..4 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0 0 advance to epoch 1 (<= last 4 <= newest_map 4) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0 1 advance to epoch 2 (<= last 4 <= newest_map 4) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0 2 advance to epoch 3 (<= last 4 <= newest_map 4) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0 3 advance to epoch 4 (<= last 4 <= newest_map 4) 2026-02-11T23:30:07.651+0000 7fac5e983640 0 osd.0 4 crush map has features 288514050185494528, adjusting msgr requires for clients 2026-02-11T23:30:07.651+0000 7fac5e983640 0 osd.0 4 crush map has features 288514050185494528 was 288232575208792577, adjusting msgr requires for mons 2026-02-11T23:30:07.651+0000 7fac5e983640 0 osd.0 4 crush map has features 3314932999778484224, adjusting msgr requires for osds 2026-02-11T23:30:07.651+0000 7fac5e983640 1 osd.0 4 check_osdmap_features require_osd_release unknown -> tentacle 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0 4 consume_map version 4 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0 0 await_reserved_maps epoch:4 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0 0 await_reserved_maps done 0 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0 4 consume_map 4 -- publish done 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.651+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 9 ==== mon_command_ack([{prefix=config set, name=osd_mclock_max_capacity_iops_ssd}]=0 v3)=0 v3) ==== 143+0+0 (crc 0 0 0) 0x38504e0 con 0x396bc00 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.651+0000 7fac626e8640 10 monclient: handle_mon_command_ack 3 [{"prefix": "config set", "who": "osd.0", "name": "osd_mclock_max_capacity_iops_ssd", "value": "67517.896294"}] 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.651+0000 7fac626e8640 10 monclient: _finish_command 3 = system:0 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0 4 prune_pg_created 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0:0.consume_map 4 (was 0) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0:1.consume_map 4 (was 0) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0:2.consume_map 4 (was 0) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0:3.consume_map 4 (was 0) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0:4.consume_map 4 (was 0) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0:5.consume_map 4 (was 0) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0:6.consume_map 4 (was 0) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0:7.consume_map 4 (was 0) 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0 4 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:07.651+0000 7fac5e983640 10 osd.0 4 _preboot _preboot mon has osdmaps 1..4 2026-02-11T23:30:07.651+0000 7fac5e983640 5 osd.0 4 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x256/0x1000, compress 0x0/0x0/0x0, omap 0x63a, meta 0x1a4f9c6), peers [] op hist []) 2026-02-11T23:30:07.651+0000 7fac5e983640 20 osd.0 4 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:07.651+0000 7fac66be7640 10 osd.0 4 operator() waiting for peering work to drain 2026-02-11T23:30:07.651+0000 7fac66be7640 10 osd.0 4 _send_boot 2026-02-11T23:30:07.651+0000 7fac66be7640 20 osd.0 4 initial client_addrs [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604], cluster_addrs [v2:0.0.0.0:6802/460666604,v1:0.0.0.0:6803/460666604], hb_back_addrs [v2:0.0.0.0:6806/460666604,v1:0.0.0.0:6807/460666604], hb_front_addrs [v2:0.0.0.0:6804/460666604,v1:0.0.0.0:6805/460666604] 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:0.0.0.0:6802/460666604,v1:0.0.0.0:6803/460666604] set_addr_unknowns [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:0.0.0.0:6802/460666604,v1:0.0.0.0:6803/460666604] set_addr_unknowns assuming my addr v2:0.0.0.0:6802/460666604 matches provided addr v2:127.0.0.1:6800/460666604 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:0.0.0.0:6802/460666604,v1:0.0.0.0:6803/460666604] set_addr_unknowns assuming my addr v1:0.0.0.0:6803/460666604 matches provided addr v2:127.0.0.1:6800/460666604 2026-02-11T23:30:07.651+0000 7fac66be7640 10 osd.0 4 new session (outgoing) 0x38e3500 con=0x2519000 addr=v2:127.0.0.1:6802/460666604 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] set_addr_unknowns now [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] 2026-02-11T23:30:07.651+0000 7fac66be7640 10 osd.0 4 assuming cluster_addrs match client_addrs [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:0.0.0.0:6806/460666604,v1:0.0.0.0:6807/460666604] set_addr_unknowns [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:0.0.0.0:6806/460666604,v1:0.0.0.0:6807/460666604] set_addr_unknowns assuming my addr v2:0.0.0.0:6806/460666604 matches provided addr v2:127.0.0.1:6802/460666604 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:0.0.0.0:6806/460666604,v1:0.0.0.0:6807/460666604] set_addr_unknowns assuming my addr v1:0.0.0.0:6807/460666604 matches provided addr v2:127.0.0.1:6802/460666604 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] set_addr_unknowns now [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] 2026-02-11T23:30:07.651+0000 7fac66be7640 10 osd.0 4 assuming hb_back_addrs match cluster_addrs [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:0.0.0.0:6804/460666604,v1:0.0.0.0:6805/460666604] set_addr_unknowns [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:0.0.0.0:6804/460666604,v1:0.0.0.0:6805/460666604] set_addr_unknowns assuming my addr v2:0.0.0.0:6804/460666604 matches provided addr v2:127.0.0.1:6800/460666604 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:0.0.0.0:6804/460666604,v1:0.0.0.0:6805/460666604] set_addr_unknowns assuming my addr v1:0.0.0.0:6805/460666604 matches provided addr v2:127.0.0.1:6800/460666604 2026-02-11T23:30:07.651+0000 7fac66be7640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] set_addr_unknowns now [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] 2026-02-11T23:30:07.651+0000 7fac66be7640 10 osd.0 4 assuming hb_front_addrs match client_addrs [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] 2026-02-11T23:30:07.651+0000 7fac66be7640 1 osd.0 4 set_numa_affinity storage numa node 0 2026-02-11T23:30:07.651+0000 7fac66be7640 -1 osd.0 4 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-11T23:30:07.651+0000 7fac66be7640 1 osd.0 4 set_numa_affinity not setting numa affinity 2026-02-11T23:30:07.651+0000 7fac66be7640 10 osd.0 4 final client_addrs [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604], cluster_addrs [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604], hb_back_addrs [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604], hb_front_addrs [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] 2026-02-11T23:30:07.651+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 10 ==== config(1 keys) ==== 56+0+0 (crc 0 0 0) 0x3919380 con 0x396bc00 2026-02-11T23:30:07.651+0000 7fac626e8640 10 monclient: handle_config config(1 keys) 2026-02-11T23:30:07.651+0000 7fac6ebf7640 4 set_mon_vals no callback set 2026-02-11T23:30:07.651+0000 7fac66be7640 1 bluestore(td/osd-backfill-space/0) collect_metadata devices span numa nodes 0 2026-02-11T23:30:07.651+0000 7fac6ebf7640 10 set_mon_vals osd_mclock_max_capacity_iops_ssd = 67517.896294 2026-02-11T23:30:07.651+0000 7fac66be7640 1 bluestore(td/osd-backfill-space/0) collect_metadata devices span numa nodes 0 2026-02-11T23:30:07.652+0000 7fac6ebf7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 18636.41 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:07.652+0000 7fac6ebf7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 18636.41 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:07.652+0000 7fac6ebf7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 18636.41 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:07.652+0000 7fac6ebf7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 18636.41 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:07.652+0000 7fac6ebf7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 18636.41 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:07.652+0000 7fac6ebf7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 18636.41 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:07.652+0000 7fac6ebf7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 18636.41 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:07.652+0000 7fac6ebf7640 1 mclock_common: set_from_config: osd_bandwidth_cost_per_io: 18636.41 bytes/io, osd_bandwidth_capacity_per_shard 157286400.00 bytes/second 2026-02-11T23:30:07.652+0000 7fac66be7640 10 osd.0 4 _collect_metadata {arch=x86_64,back_addr=[v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604],back_iface=,bluefs=1,bluefs_dedicated_db=0,bluefs_dedicated_wal=0,bluefs_single_shared_device=1,bluestore_allocation_from_file=1,bluestore_allocator=hybrid,bluestore_bdev_access_mode=file,bluestore_bdev_block_size=4096,bluestore_bdev_devices=nvme0n1,bluestore_bdev_driver=KernelDevice,bluestore_bdev_optimal_io_size=4096,bluestore_bdev_path=td/osd-backfill-space/0/block,bluestore_bdev_rotational=0,bluestore_bdev_size=107374182400,bluestore_bdev_support_discard=1,bluestore_bdev_type=ssd,bluestore_min_alloc_size=4096,bluestore_onode_segmentation=inactive,bluestore_write_mode=classic,ceph_release=tentacle,ceph_version=ceph version 20.3.0-5256-gf2a98edf (f2a98edfc002a9c9c8df236a812719da23bb2dd0) tentacle (dev - RelWithDebInfo),ceph_version_short=20.3.0-5256-gf2a98edf,ceph_version_when_created=ceph version 20.3.0-5256-gf2a98edf (f2a98edfc002a9c9c8df236a812719da23bb2dd0) tentacle (dev - RelWithDebInfo),cpu=AMD EPYC 4464P 12-Core Processor,created_at=2026-02-11T23:30:06.407367Z,default_device_class=ssd,device_ids=nvme0n1=Micron_7450_MTFDKBA960TFR_25174FB04F5D,device_paths=nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1,devices=nvme0n1,distro=centos,distro_description=CentOS Stream 9,distro_version=9,front_addr=[v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604],front_iface=,hb_back_addr=[v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604],hb_front_addr=[v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604],hostname=trial114,journal_rotational=0,kernel_description=#1 SMP PREEMPT_DYNAMIC Fri Feb 6 13:57:07 UTC 2026,kernel_version=5.14.0-677.el9.x86_64,mem_swap_kb=8388604,mem_total_kb=130438948,network_numa_unknown_ifaces=back_iface,front_iface,objectstore_numa_node=0,objectstore_numa_nodes=0,os=Linux,osd_data=td/osd-backfill-space/0,osd_objectstore=bluestore,osdspec_affinity=,rotational=0} 2026-02-11T23:30:07.652+0000 7fac66be7640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:07.652+0000 7fac66be7640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- osd_boot(osd.0 booted 0 features 4544132024016699391 v4) -- 0x389ce00 con 0x396bc00 2026-02-11T23:30:07.751+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 11 ==== osd_map(4..5 src has 1..5) ==== 1489+0+0 (crc 0 0 0) 0x34bfc00 con 0x396bc00 2026-02-11T23:30:07.751+0000 7fac626e8640 20 osd.0 4 OSD::ms_dispatch: osd_map(4..5 src has 1..5) 2026-02-11T23:30:07.751+0000 7fac626e8640 20 osd.0 4 _dispatch 0x34bfc00 osd_map(4..5 src has 1..5) 2026-02-11T23:30:07.751+0000 7fac626e8640 10 monclient: _renew_subs 2026-02-11T23:30:07.751+0000 7fac626e8640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:07.751+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=0}) -- 0x38504e0 con 0x396bc00 2026-02-11T23:30:07.751+0000 7fac626e8640 3 osd.0 4 handle_osd_map epochs [4,5], i have 4, src has [1,5] 2026-02-11T23:30:07.751+0000 7fac626e8640 10 osd.0 4 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:07.751+0000 7fac626e8640 10 osd.0 4 handle_osd_map got inc map for epoch 5 2026-02-11T23:30:07.751+0000 7fac626e8640 20 osd.0 4 got_full_map 5, nothing requested 2026-02-11T23:30:07.751+0000 7fac626e8640 20 osd.0 4 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:07.751+0000 7fac626e8640 20 osd.0 4 handle_osd_map pg_num_history pg_num_history(e5 pg_nums {} deleted_pools ) 2026-02-11T23:30:07.751+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {5={}} 2026-02-11T23:30:07.751+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:07.751+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e5 maps [1~5] lci=[0,0] tlb=1) 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0 4 _committed_osd_maps 5..5 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0 4 advance to epoch 5 (<= last 5 <= newest_map 5) 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0 5 up_epoch is 5 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0 5 boot_epoch is 5 2026-02-11T23:30:07.751+0000 7fac5e983640 1 osd.0 5 state: booting -> active 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0 5 consume_map version 5 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0 4 await_reserved_maps epoch:5 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0 4 await_reserved_maps done 0 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0 5 consume_map 5 -- publish done 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.751+0000 7fac5e983640 20 osd.0 5 prune_pg_created 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0:0.consume_map 5 (was 4) 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0:1.consume_map 5 (was 4) 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0:2.consume_map 5 (was 4) 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0:3.consume_map 5 (was 4) 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0:4.consume_map 5 (was 4) 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0:5.consume_map 5 (was 4) 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0:6.consume_map 5 (was 4) 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0:7.consume_map 5 (was 4) 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0 5 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0 5 maybe_update_heartbeat_peers updating 2026-02-11T23:30:07.751+0000 7fac5e983640 10 osd.0 5 maybe_update_heartbeat_peers 0 peers, extras 2026-02-11T23:30:07.751+0000 7fac5e983640 7 osd.0 5 activate_map version 5 2026-02-11T23:30:07.751+0000 7fac55971640 20 osd.0 5 agent_entry empty queue 2026-02-11T23:30:07.752+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 12 ==== osd_map(5..5 src has 1..5) ==== 662+0+0 (crc 0 0 0) 0x38e36c0 con 0x396bc00 2026-02-11T23:30:07.752+0000 7fac626e8640 20 osd.0 5 OSD::ms_dispatch: osd_map(5..5 src has 1..5) 2026-02-11T23:30:07.752+0000 7fac626e8640 20 osd.0 5 _dispatch 0x38e36c0 osd_map(5..5 src has 1..5) 2026-02-11T23:30:07.752+0000 7fac626e8640 10 monclient: _renew_subs 2026-02-11T23:30:07.752+0000 7fac626e8640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:07.752+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=0}) -- 0x3850d00 con 0x396bc00 2026-02-11T23:30:07.752+0000 7fac626e8640 3 osd.0 5 handle_osd_map epochs [5,5], i have 5, src has [1,5] 2026-02-11T23:30:07.752+0000 7fac626e8640 10 osd.0 5 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:07.752+0000 7fac626e8640 10 osd.0 5 no new maps here, dropping 2026-02-11T23:30:07.752+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 13 ==== osd_map(5..5 src has 1..5) ==== 1299+0+0 (crc 0 0 0) 0x38e2fc0 con 0x396bc00 2026-02-11T23:30:07.752+0000 7fac626e8640 20 osd.0 5 OSD::ms_dispatch: osd_map(5..5 src has 1..5) 2026-02-11T23:30:07.752+0000 7fac626e8640 20 osd.0 5 _dispatch 0x38e2fc0 osd_map(5..5 src has 1..5) 2026-02-11T23:30:07.752+0000 7fac626e8640 3 osd.0 5 handle_osd_map epochs [5,5], i have 5, src has [1,5] 2026-02-11T23:30:07.752+0000 7fac626e8640 10 osd.0 5 handle_osd_map: received 1 full maps and 0 incremental maps 2026-02-11T23:30:07.752+0000 7fac626e8640 10 osd.0 5 no new maps here, dropping 2026-02-11T23:30:07.752+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 14 ==== osd_map(5..5 src has 1..5) ==== 1299+0+0 (crc 0 0 0) 0x34bfc00 con 0x396bc00 2026-02-11T23:30:07.752+0000 7fac626e8640 20 osd.0 5 OSD::ms_dispatch: osd_map(5..5 src has 1..5) 2026-02-11T23:30:07.752+0000 7fac626e8640 20 osd.0 5 _dispatch 0x34bfc00 osd_map(5..5 src has 1..5) 2026-02-11T23:30:07.752+0000 7fac626e8640 3 osd.0 5 handle_osd_map epochs [5,5], i have 5, src has [1,5] 2026-02-11T23:30:07.752+0000 7fac626e8640 10 osd.0 5 handle_osd_map: received 1 full maps and 0 incremental maps 2026-02-11T23:30:07.752+0000 7fac626e8640 10 osd.0 5 no new maps here, dropping 2026-02-11T23:30:07.906+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 15 ==== mgrmap(e 2) ==== 53625+0+0 (crc 0 0 0) 0x458c600 con 0x396bc00 2026-02-11T23:30:07.997+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 16 ==== osd_map(5..6 src has 1..6) ==== 950+0+0 (crc 0 0 0) 0x38e36c0 con 0x396bc00 2026-02-11T23:30:07.997+0000 7fac626e8640 20 osd.0 5 OSD::ms_dispatch: osd_map(5..6 src has 1..6) 2026-02-11T23:30:07.997+0000 7fac626e8640 20 osd.0 5 _dispatch 0x38e36c0 osd_map(5..6 src has 1..6) 2026-02-11T23:30:07.997+0000 7fac626e8640 3 osd.0 5 handle_osd_map epochs [5,6], i have 5, src has [1,6] 2026-02-11T23:30:07.997+0000 7fac626e8640 10 osd.0 5 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:07.997+0000 7fac626e8640 10 osd.0 5 handle_osd_map got inc map for epoch 6 2026-02-11T23:30:07.997+0000 7fac626e8640 20 osd.0 5 got_full_map 6, nothing requested 2026-02-11T23:30:07.997+0000 7fac626e8640 20 osd.0 5 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:07.997+0000 7fac626e8640 20 osd.0 5 handle_osd_map pg_num_history pg_num_history(e6 pg_nums {} deleted_pools ) 2026-02-11T23:30:07.997+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {6={}} 2026-02-11T23:30:07.997+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:07.997+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e6 maps [1~6] lci=[5,6] tlb=1) 2026-02-11T23:30:07.997+0000 7fac5e983640 10 osd.0 5 _committed_osd_maps 6..6 2026-02-11T23:30:07.997+0000 7fac5e983640 10 osd.0 5 advance to epoch 6 (<= last 6 <= newest_map 6) 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0 6 consume_map version 6 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0 5 await_reserved_maps epoch:6 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0 5 await_reserved_maps done 0 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0 6 consume_map 6 -- publish done 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:07.997+0000 7fac5e983640 20 osd.0 6 prune_pg_created 2026-02-11T23:30:07.997+0000 7fac5e983640 10 osd.0:0.consume_map 6 (was 5) 2026-02-11T23:30:07.997+0000 7fac5e983640 10 osd.0:1.consume_map 6 (was 5) 2026-02-11T23:30:07.997+0000 7fac5e983640 10 osd.0:2.consume_map 6 (was 5) 2026-02-11T23:30:07.997+0000 7fac5e983640 10 osd.0:3.consume_map 6 (was 5) 2026-02-11T23:30:07.997+0000 7fac5e983640 10 osd.0:4.consume_map 6 (was 5) 2026-02-11T23:30:07.997+0000 7fac5e983640 10 osd.0:5.consume_map 6 (was 5) 2026-02-11T23:30:07.997+0000 7fac5e983640 10 osd.0:6.consume_map 6 (was 5) 2026-02-11T23:30:07.997+0000 7fac5e983640 10 osd.0:7.consume_map 6 (was 5) 2026-02-11T23:30:07.997+0000 7fac5e983640 10 osd.0 6 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:07.997+0000 7fac5e983640 7 osd.0 6 activate_map version 6 2026-02-11T23:30:07.997+0000 7fac55971640 20 osd.0 6 agent_entry empty queue 2026-02-11T23:30:08.007+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 17 ==== mgrmap(e 3) ==== 53752+0+0 (crc 0 0 0) 0x4470000 con 0x396bc00 2026-02-11T23:30:08.007+0000 7fac626e8640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] conn(0x396a800 0x38a6680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:08.008+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] conn(0x396a800 0x38a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:08.008+0000 7fac703fa640 10 monclient: get_auth_request con 0x396a800 auth_method 0 2026-02-11T23:30:08.008+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgropen(unknown.0) -- 0x4459680 con 0x396a800 2026-02-11T23:30:08.008+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] conn(0x396a800 0x38a6680 crc :-1 s=READY pgs=1 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:08.008+0000 7fac626e8640 10 osd.0 6 ms_handle_connect con 0x396a800 2026-02-11T23:30:08.009+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mgr.4100 v2:127.0.0.1:6808/954425105 1 ==== mgrconfigure(period=5, threshold=5) ==== 13+0+0 (crc 0 0 0) 0x34bfc00 con 0x396a800 2026-02-11T23:30:08.009+0000 7fac626e8640 20 osd.0 6 reports for 0 queries 2026-02-11T23:30:08.009+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +277-0 packed 3294 task_status=0) -- 0x389d180 con 0x396a800 2026-02-11T23:30:08.009+0000 7fac626e8640 15 osd.0 6 collect_pg_stats 2026-02-11T23:30:08.009+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836481 v 0) -- 0x4470000 con 0x396a800 2026-02-11T23:30:08.149+0000 7fac683ea640 10 osd.0 6 tick_without_osd_lock 2026-02-11T23:30:08.149+0000 7fac683ea640 0 osd.0 6 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:08.149+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:08.149+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:08.149+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:08.149+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.093 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:08.149+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:08.149, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:08.149+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:08.149+0000 7fac683ea640 10 osd.0 6 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:08.149+0000 7fac683ea640 20 osd.0 6 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:08.149+0000 7fac683ea640 10 osd.0 6 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:08.149+0000 7fac683ea640 20 osd.0 6 send_beacon sending 2026-02-11T23:30:08.149+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:08.149+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- osd_beacon(pgs [] lec 6 last_purged_snaps_scrub 0.000000 osd_beacon_report_interval 300 v6) -- 0x3b1a000 con 0x396bc00 2026-02-11T23:30:08.160+0000 7fac68beb640 10 osd.0 6 tick 2026-02-11T23:30:08.160+0000 7fac68beb640 20 osd.0 6 tick last_purged_snaps_scrub 0.000000 next 115256.250000 ... now 2026-02-11T23:30:08.160+0000 7fac68beb640 10 osd.0 6 scrub_purged_snaps 2026-02-11T23:30:08.160+0000 7fac68beb640 0 log_channel(cluster) log [DBG] : purged_snaps scrub starts 2026-02-11T23:30:08.160+0000 7fac68beb640 10 snap_mapper run 2026-02-11T23:30:08.160+0000 7fac68beb640 10 snap_mapper run end, found 0 stray 2026-02-11T23:30:08.160+0000 7fac68beb640 0 log_channel(cluster) log [DBG] : purged_snaps scrub ok 2026-02-11T23:30:08.160+0000 7fac68beb640 10 osd.0 6 scrub_purged_snaps done queueing pgs, updating superblock 2026-02-11T23:30:08.160+0000 7fac68beb640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e6 maps [1~6] lci=[5,6] tlb=1) 2026-02-11T23:30:08.160+0000 7fac68beb640 20 osd.0 6 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:08.160+0000 7fac68beb640 10 osd.0 6 scrub_purged_snaps done 2026-02-11T23:30:09.137+0000 7fac68beb640 10 osd.0 6 tick 2026-02-11T23:30:09.137+0000 7fac68beb640 20 osd.0 6 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:09.195+0000 7fac683ea640 10 osd.0 6 tick_without_osd_lock 2026-02-11T23:30:09.195+0000 7fac683ea640 0 osd.0 6 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:09.195+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:09.195+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:09.195+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:09.195+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.093 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:09.195+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:09.195, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:09.195+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:09.195+0000 7fac683ea640 10 osd.0 6 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:09.195+0000 7fac683ea640 20 osd.0 6 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:09.195+0000 7fac683ea640 10 osd.0 6 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:09.195+0000 7fac683ea640 20 osd.0 6 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:09.210+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 18 ==== osd_map(6..7 src has 1..7) ==== 1172+0+0 (crc 0 0 0) 0x38e2fc0 con 0x396bc00 2026-02-11T23:30:09.210+0000 7fac626e8640 20 osd.0 6 OSD::ms_dispatch: osd_map(6..7 src has 1..7) 2026-02-11T23:30:09.210+0000 7fac626e8640 20 osd.0 6 _dispatch 0x38e2fc0 osd_map(6..7 src has 1..7) 2026-02-11T23:30:09.210+0000 7fac626e8640 3 osd.0 6 handle_osd_map epochs [6,7], i have 6, src has [1,7] 2026-02-11T23:30:09.210+0000 7fac626e8640 10 osd.0 6 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:09.210+0000 7fac626e8640 10 osd.0 6 handle_osd_map got inc map for epoch 7 2026-02-11T23:30:09.210+0000 7fac626e8640 20 osd.0 6 got_full_map 7, nothing requested 2026-02-11T23:30:09.210+0000 7fac626e8640 20 osd.0 6 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:09.210+0000 7fac626e8640 20 osd.0 6 handle_osd_map pg_num_history pg_num_history(e7 pg_nums {} deleted_pools ) 2026-02-11T23:30:09.210+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {7={}} 2026-02-11T23:30:09.210+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:09.210+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e7 maps [1~7] lci=[5,7] tlb=1) 2026-02-11T23:30:09.210+0000 7fac5e983640 10 osd.0 6 _committed_osd_maps 7..7 2026-02-11T23:30:09.210+0000 7fac5e983640 10 osd.0 6 advance to epoch 7 (<= last 7 <= newest_map 7) 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0 7 consume_map version 7 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0 6 await_reserved_maps epoch:7 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0 6 await_reserved_maps done 0 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0 7 consume_map 7 -- publish done 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.210+0000 7fac5e983640 20 osd.0 7 prune_pg_created 2026-02-11T23:30:09.210+0000 7fac5e983640 10 osd.0:0.consume_map 7 (was 6) 2026-02-11T23:30:09.210+0000 7fac5e983640 10 osd.0:1.consume_map 7 (was 6) 2026-02-11T23:30:09.210+0000 7fac5e983640 10 osd.0:2.consume_map 7 (was 6) 2026-02-11T23:30:09.210+0000 7fac5e983640 10 osd.0:3.consume_map 7 (was 6) 2026-02-11T23:30:09.210+0000 7fac5e983640 10 osd.0:4.consume_map 7 (was 6) 2026-02-11T23:30:09.210+0000 7fac5e983640 10 osd.0:5.consume_map 7 (was 6) 2026-02-11T23:30:09.210+0000 7fac5e983640 10 osd.0:6.consume_map 7 (was 6) 2026-02-11T23:30:09.210+0000 7fac5e983640 10 osd.0:7.consume_map 7 (was 6) 2026-02-11T23:30:09.210+0000 7fac5e983640 10 osd.0 7 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:09.210+0000 7fac5e983640 7 osd.0 7 activate_map version 7 2026-02-11T23:30:09.210+0000 7fac55971640 20 osd.0 7 agent_entry empty queue 2026-02-11T23:30:09.311+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 19 ==== osd_map(7..8 src has 1..8) ==== 1748+0+0 (crc 0 0 0) 0x38e3180 con 0x396bc00 2026-02-11T23:30:09.311+0000 7fac626e8640 20 osd.0 7 OSD::ms_dispatch: osd_map(7..8 src has 1..8) 2026-02-11T23:30:09.311+0000 7fac626e8640 20 osd.0 7 _dispatch 0x38e3180 osd_map(7..8 src has 1..8) 2026-02-11T23:30:09.311+0000 7fac626e8640 3 osd.0 7 handle_osd_map epochs [7,8], i have 7, src has [1,8] 2026-02-11T23:30:09.311+0000 7fac626e8640 10 osd.0 7 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:09.311+0000 7fac626e8640 10 osd.0 7 handle_osd_map got inc map for epoch 8 2026-02-11T23:30:09.311+0000 7fac626e8640 20 osd.0 7 got_full_map 8, nothing requested 2026-02-11T23:30:09.311+0000 7fac626e8640 20 osd.0 7 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:09.311+0000 7fac626e8640 20 osd.0 7 handle_osd_map pg_num_history pg_num_history(e8 pg_nums {} deleted_pools ) 2026-02-11T23:30:09.311+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {8={}} 2026-02-11T23:30:09.311+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:09.311+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e8 maps [1~8] lci=[5,8] tlb=1) 2026-02-11T23:30:09.311+0000 7fac5e983640 10 osd.0 7 _committed_osd_maps 8..8 2026-02-11T23:30:09.311+0000 7fac5e983640 10 osd.0 7 advance to epoch 8 (<= last 8 <= newest_map 8) 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0 8 consume_map version 8 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0 7 await_reserved_maps epoch:8 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0 7 await_reserved_maps done 0 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0 8 consume_map 8 -- publish done 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:09.311+0000 7fac5e983640 20 osd.0 8 prune_pg_created 2026-02-11T23:30:09.311+0000 7fac5e983640 10 osd.0:0.consume_map 8 (was 7) 2026-02-11T23:30:09.311+0000 7fac5e983640 10 osd.0:1.consume_map 8 (was 7) 2026-02-11T23:30:09.311+0000 7fac5e983640 10 osd.0:2.consume_map 8 (was 7) 2026-02-11T23:30:09.311+0000 7fac5e983640 10 osd.0:3.consume_map 8 (was 7) 2026-02-11T23:30:09.311+0000 7fac5e983640 10 osd.0:4.consume_map 8 (was 7) 2026-02-11T23:30:09.311+0000 7fac5e983640 10 osd.0:5.consume_map 8 (was 7) 2026-02-11T23:30:09.311+0000 7fac5e983640 10 osd.0:6.consume_map 8 (was 7) 2026-02-11T23:30:09.311+0000 7fac5e983640 10 osd.0:7.consume_map 8 (was 7) 2026-02-11T23:30:09.311+0000 7fac5e983640 10 osd.0 8 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:09.311+0000 7fac5e983640 7 osd.0 8 activate_map version 8 2026-02-11T23:30:09.311+0000 7fac55971640 20 osd.0 8 agent_entry empty queue 2026-02-11T23:30:09.732+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> conn(0x396a400 0x26d5080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:09.732+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> conn(0x396a000 0x38a6c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:09.732+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> conn(0x396a400 0x26d5080 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:09.732+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> conn(0x396a000 0x38a6c00 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:09.732+0000 7fac71bfd640 10 osd.0 8 ms_handle_fast_authentication new session 0x38e36c0 con 0x396a400 entity osd.1 addr 2026-02-11T23:30:09.732+0000 7fac723fe640 10 osd.0 8 ms_handle_fast_authentication new session 0x38e2e00 con 0x396a000 entity osd.1 addr 2026-02-11T23:30:09.732+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> 127.0.0.1:0/4244365097 conn(0x396a400 0x26d5080 crc :-1 s=READY pgs=1 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:09.732+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> 127.0.0.1:0/4244365097 conn(0x396a000 0x38a6c00 crc :-1 s=READY pgs=1 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:09.935+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 20 ==== mgrmap(e 4) ==== 53858+0+0 (crc 0 0 0) 0x4470300 con 0x396bc00 2026-02-11T23:30:10.068+0000 7fac4d160640 5 osd.0 8 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x274d/0xe000, compress 0x0/0x0/0x0, omap 0x1081, meta 0x1a4ef7f), peers [] op hist []) 2026-02-11T23:30:10.068+0000 7fac4d160640 20 osd.0 8 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:10.068+0000 7fac4d160640 10 osd.0 8 i have no heartbeat peers; checking mon for new map 2026-02-11T23:30:10.068+0000 7fac4d160640 10 monclient: _renew_subs 2026-02-11T23:30:10.068+0000 7fac4d160640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:10.068+0000 7fac4d160640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=9}) -- 0x44a61a0 con 0x396bc00 2026-02-11T23:30:10.068+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 21 ==== osd_map(9..9 src has 1..9) ==== 662+0+0 (crc 0 0 0) 0x38e2fc0 con 0x396bc00 2026-02-11T23:30:10.068+0000 7fac626e8640 20 osd.0 8 OSD::ms_dispatch: osd_map(9..9 src has 1..9) 2026-02-11T23:30:10.068+0000 7fac626e8640 20 osd.0 8 _dispatch 0x38e2fc0 osd_map(9..9 src has 1..9) 2026-02-11T23:30:10.068+0000 7fac626e8640 3 osd.0 8 handle_osd_map epochs [9,9], i have 8, src has [1,9] 2026-02-11T23:30:10.068+0000 7fac626e8640 10 osd.0 8 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:10.068+0000 7fac626e8640 10 osd.0 8 handle_osd_map got inc map for epoch 9 2026-02-11T23:30:10.068+0000 7fac626e8640 20 osd.0 8 got_full_map 9, nothing requested 2026-02-11T23:30:10.068+0000 7fac626e8640 20 osd.0 8 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:10.068+0000 7fac626e8640 20 osd.0 8 handle_osd_map pg_num_history pg_num_history(e9 pg_nums {} deleted_pools ) 2026-02-11T23:30:10.068+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {9={}} 2026-02-11T23:30:10.068+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:10.068+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e9 maps [1~9] lci=[5,9] tlb=1) 2026-02-11T23:30:10.068+0000 7fac5e983640 10 osd.0 8 _committed_osd_maps 9..9 2026-02-11T23:30:10.068+0000 7fac5e983640 10 osd.0 8 advance to epoch 9 (<= last 9 <= newest_map 9) 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0 9 consume_map version 9 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0 8 await_reserved_maps epoch:9 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0 8 await_reserved_maps done 0 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0 9 consume_map 9 -- publish done 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.068+0000 7fac5e983640 20 osd.0 9 prune_pg_created 2026-02-11T23:30:10.068+0000 7fac5e983640 10 osd.0:0.consume_map 9 (was 8) 2026-02-11T23:30:10.068+0000 7fac5e983640 10 osd.0:1.consume_map 9 (was 8) 2026-02-11T23:30:10.069+0000 7fac5e983640 10 osd.0:2.consume_map 9 (was 8) 2026-02-11T23:30:10.069+0000 7fac5e983640 10 osd.0:3.consume_map 9 (was 8) 2026-02-11T23:30:10.069+0000 7fac5e983640 10 osd.0:4.consume_map 9 (was 8) 2026-02-11T23:30:10.069+0000 7fac5e983640 10 osd.0:5.consume_map 9 (was 8) 2026-02-11T23:30:10.069+0000 7fac5e983640 10 osd.0:6.consume_map 9 (was 8) 2026-02-11T23:30:10.069+0000 7fac5e983640 10 osd.0:7.consume_map 9 (was 8) 2026-02-11T23:30:10.069+0000 7fac5e983640 10 osd.0 9 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:10.069+0000 7fac5e983640 10 osd.0 9 maybe_update_heartbeat_peers updating 2026-02-11T23:30:10.069+0000 7fac5e983640 10 osd.0 9 adding neighbor peer osd.1 2026-02-11T23:30:10.069+0000 7fac5e983640 20 osd.0 9 get_con_osd_hb to osd.1 from_epoch 9 2026-02-11T23:30:10.069+0000 7fac5e983640 20 osd.0 9 get_nextmap_reserved map_reservations: {9=1} 2026-02-11T23:30:10.069+0000 7fac5e983640 1 --2- >> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] conn(0x396b400 0x3eda000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:10.069+0000 7fac5e983640 1 --2- >> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] conn(0x3752400 0x3eda580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:10.069+0000 7fac5e983640 20 osd.0 9 release_map epoch: 9 2026-02-11T23:30:10.069+0000 7fac5e983640 10 osd.0 9 _add_heartbeat_peer: new peer osd.1 v2:127.0.0.1:6816/4244365097 v2:127.0.0.1:6814/4244365097 2026-02-11T23:30:10.069+0000 7fac5e983640 10 osd.0 9 maybe_update_heartbeat_peers 1 peers, extras 1 2026-02-11T23:30:10.069+0000 7fac5e983640 7 osd.0 9 activate_map version 9 2026-02-11T23:30:10.069+0000 7fac55971640 20 osd.0 9 agent_entry empty queue 2026-02-11T23:30:10.069+0000 7fac703fa640 1 --2- >> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] conn(0x396b400 0x3eda000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:10.069+0000 7fac703fa640 1 --2- >> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] conn(0x396b400 0x3eda000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:127.0.0.1:6816/4244365097 says I am v2:127.0.0.1:57502/0 (socket says 127.0.0.1:57502) 2026-02-11T23:30:10.069+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 learned_addr learned my addr 127.0.0.1:0/460666604 (peer_addr_for_me v2:127.0.0.1:0/0) 2026-02-11T23:30:10.069+0000 7fac703fa640 10 monclient: get_auth_request con 0x396b400 auth_method 0 2026-02-11T23:30:10.069+0000 7fac703fa640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] conn(0x396b400 0x3eda000 crc :-1 s=READY pgs=1 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:10.078+0000 7fac71bfd640 1 --2- >> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] conn(0x3752400 0x3eda580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:10.078+0000 7fac71bfd640 1 --2- >> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] conn(0x3752400 0x3eda580 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:127.0.0.1:6814/4244365097 says I am v2:127.0.0.1:59284/0 (socket says 127.0.0.1:59284) 2026-02-11T23:30:10.078+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 learned_addr learned my addr 127.0.0.1:0/460666604 (peer_addr_for_me v2:127.0.0.1:0/0) 2026-02-11T23:30:10.078+0000 7fac71bfd640 10 monclient: get_auth_request con 0x3752400 auth_method 0 2026-02-11T23:30:10.078+0000 7fac71bfd640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] conn(0x3752400 0x3eda580 crc :-1 s=READY pgs=1 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:10.100+0000 7fac68beb640 10 osd.0 9 tick 2026-02-11T23:30:10.100+0000 7fac68beb640 20 osd.0 9 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:10.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:10.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:10.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:40.167550+0000) 2026-02-11T23:30:10.167+0000 7fac616e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:10.167+0000 7fac616e6640 10 log_client log_queue is 2 last_log 2 sent 0 num 2 unsent 2 sending 2 2026-02-11T23:30:10.167+0000 7fac616e6640 10 log_client will send 2026-02-11T23:30:08.161290+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2026-02-11T23:30:10.167+0000 7fac616e6640 10 log_client will send 2026-02-11T23:30:08.161321+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2026-02-11T23:30:10.167+0000 7fac616e6640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:10.167+0000 7fac616e6640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- log(2 entries from seq 1 at 2026-02-11T23:30:08.161290+0000) -- 0x38e2000 con 0x396bc00 2026-02-11T23:30:10.217+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 22 ==== log(last 2) ==== 28+0+0 (crc 0 0 0) 0x3b1a000 con 0x396bc00 2026-02-11T23:30:10.217+0000 7fac626e8640 10 log_client handle_log_ack log(last 2) 2026-02-11T23:30:10.217+0000 7fac626e8640 10 log_client logged 2026-02-11T23:30:08.161290+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2026-02-11T23:30:10.217+0000 7fac626e8640 10 log_client logged 2026-02-11T23:30:08.161321+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2026-02-11T23:30:10.223+0000 7fac683ea640 10 osd.0 9 tick_without_osd_lock 2026-02-11T23:30:10.223+0000 7fac683ea640 0 osd.0 9 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:10.223+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:10.223+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:10.223+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:10.223+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.196 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:10.223+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:10.224, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:10.223+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:10.223+0000 7fac683ea640 10 osd.0 9 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:10.223+0000 7fac683ea640 20 osd.0 9 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:10.223+0000 7fac683ea640 10 osd.0 9 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:10.223+0000 7fac683ea640 20 osd.0 9 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:10.568+0000 7fac4d160640 5 osd.0 9 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x41dc/0x14000, compress 0x0/0x0/0x0, omap 0x1ab1, meta 0x1a4e54f), peers [1] op hist []) 2026-02-11T23:30:10.568+0000 7fac4d160640 20 osd.0 9 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:10.568+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e9 up_from 5 ping_stamp 2026-02-11T23:30:10.568731+0000/3.403146982s send_stamp 3.403146982s) -- 0x3b1a300 con 0x396b400 2026-02-11T23:30:10.568+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e9 up_from 5 ping_stamp 2026-02-11T23:30:10.568731+0000/3.403146982s send_stamp 3.403146982s) -- 0x3b1a480 con 0x3752400 2026-02-11T23:30:10.568+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 1 ==== osd_ping(ping_reply e9 up_from 9 ping_stamp 2026-02-11T23:30:10.568731+0000/3.403146982s send_stamp 1.422407269s delta_ub -1.980739594s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a300 con 0x396b400 2026-02-11T23:30:10.568+0000 7fac703fa640 20 osd.0 9 get_con_osd_cluster to osd.1 from_epoch 9 2026-02-11T23:30:10.568+0000 7fac703fa640 20 osd.0 9 get_nextmap_reserved map_reservations: {9=1} 2026-02-11T23:30:10.568+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] conn(0x3752000 0x3edb080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:10.568+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 1 ==== osd_ping(ping_reply e9 up_from 9 ping_stamp 2026-02-11T23:30:10.568731+0000/3.403146982s send_stamp 1.422440886s delta_ub -1.980706096s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a480 con 0x3752400 2026-02-11T23:30:10.568+0000 7fac703fa640 20 osd.0 9 release_map epoch: 9 2026-02-11T23:30:10.568+0000 7fac703fa640 20 osd.0 9 handle_osd_ping new stamps hbstamp(osd.1 up_from 0 peer_clock_delta [-1.980925560s,-1.980739594s]) 2026-02-11T23:30:10.568+0000 7fac71bfd640 20 osd.0 9 get_con_osd_cluster to osd.1 from_epoch 9 2026-02-11T23:30:10.568+0000 7fac71bfd640 20 osd.0 9 get_nextmap_reserved map_reservations: {9=1} 2026-02-11T23:30:10.568+0000 7fac71bfd640 20 osd.0 9 release_map epoch: 9 2026-02-11T23:30:10.568+0000 7fac71bfd640 20 osd.0 9 handle_osd_ping new stamps hbstamp(osd.1 up_from 0 peer_clock_delta [-1.980927229s,-1.980706096s]) 2026-02-11T23:30:10.568+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> conn(0x396b000 0x38a5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:10.568+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> conn(0x396b000 0x38a5b80 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:10.568+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] conn(0x3752000 0x3edb080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:10.568+0000 7fac703fa640 10 osd.0 9 ms_handle_fast_authentication new session 0x38e3880 con 0x396b000 entity osd.1 addr 2026-02-11T23:30:10.568+0000 7fac723fe640 10 monclient: get_auth_request con 0x3752000 auth_method 0 2026-02-11T23:30:10.568+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] conn(0x396b000 0x38a5b80 crc :-1 s=SESSION_ACCEPTING pgs=1 gs=0 cs=0 l=0 c_cookie=12859581267707688682 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_existing_connection connection race detected, replacing existing=0x3752000 socket by this connection's socket 2026-02-11T23:30:10.568+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] conn(0x396b000 0x38a5b80 crc :-1 s=SESSION_ACCEPTING pgs=1 gs=0 cs=0 l=0 c_cookie=12859581267707688682 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:10.568+0000 7fac606e4640 2 osd.0 9 ms_handle_reset con 0x396b000 session 0x38e3880 2026-02-11T23:30:10.568+0000 7fac703fa640 10 osd.0 9 new session (incoming)0x38e3a40 con=0x3752000 addr=v2:127.0.0.1:6812/4244365097 must have raced with connect 2026-02-11T23:30:10.568+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] conn(0x3752000 0x3edb080 unknown :-1 s=READY pgs=1 gs=2 cs=0 l=0 c_cookie=12859581267707688682 s_cookie=5239351759992546807 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=b2766907adfbf6ea server_cookie=48b5eaaa9bac65f7 in_seq=0 out_seq=0 2026-02-11T23:30:10.848+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 1 ==== osd_ping(ping e10 up_from 9 ping_stamp 2026-02-11T23:30:10.848918+0000/1.702483058s send_stamp 1.702483058s delta_ub -1.980706096s) ==== 2033+0+0 (crc 0 0 0) 0x3919200 con 0x396a000 2026-02-11T23:30:10.848+0000 7fac723fe640 20 osd.0 9 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980954409s,-1.980706096s]) 2026-02-11T23:30:10.848+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e9 up_from 5 ping_stamp 2026-02-11T23:30:10.848918+0000/1.702483058s send_stamp 3.683437347s delta_ub 1.980954409s) -- 0x3919980 con 0x396a000 2026-02-11T23:30:10.848+0000 7fac723fe640 20 osd.0 9 get_con_osd_cluster to osd.1 from_epoch 9 2026-02-11T23:30:10.848+0000 7fac723fe640 20 osd.0 9 get_nextmap_reserved map_reservations: {9=1} 2026-02-11T23:30:10.848+0000 7fac723fe640 20 osd.0 9 release_map epoch: 9 2026-02-11T23:30:10.848+0000 7fac723fe640 10 osd.0 9 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 0 to ping map epoch of 10 2026-02-11T23:30:10.848+0000 7fac723fe640 10 osd.0 9 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 9 is not newer than session's projected_epoch of 10 2026-02-11T23:30:10.848+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 1 ==== osd_ping(ping e10 up_from 9 ping_stamp 2026-02-11T23:30:10.848918+0000/1.702483058s send_stamp 1.702483058s delta_ub -1.980706096s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a480 con 0x396a400 2026-02-11T23:30:10.848+0000 7fac71bfd640 20 osd.0 9 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980984926s,-1.980706096s]) 2026-02-11T23:30:10.848+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e9 up_from 5 ping_stamp 2026-02-11T23:30:10.848918+0000/1.702483058s send_stamp 3.683468103s delta_ub 1.980984926s) -- 0x3b1a780 con 0x396a400 2026-02-11T23:30:10.848+0000 7fac71bfd640 20 osd.0 9 get_con_osd_cluster to osd.1 from_epoch 9 2026-02-11T23:30:10.848+0000 7fac71bfd640 20 osd.0 9 get_nextmap_reserved map_reservations: {9=1} 2026-02-11T23:30:10.848+0000 7fac71bfd640 20 osd.0 9 release_map epoch: 9 2026-02-11T23:30:10.848+0000 7fac71bfd640 10 osd.0 9 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 9 is not newer than session's projected_epoch of 10 2026-02-11T23:30:10.848+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1 ==== osd_map(10..10 src has 1..10) ==== 324+0+0 (unknown 0 0 0) 0x38e3c00 con 0x3752000 2026-02-11T23:30:10.848+0000 7fac606e4640 20 osd.0 9 OSD::ms_dispatch: osd_map(10..10 src has 1..10) 2026-02-11T23:30:10.848+0000 7fac606e4640 20 osd.0 9 _dispatch 0x38e3c00 osd_map(10..10 src has 1..10) 2026-02-11T23:30:10.848+0000 7fac606e4640 3 osd.0 9 handle_osd_map epochs [10,10], i have 9, src has [1,10] 2026-02-11T23:30:10.848+0000 7fac606e4640 10 osd.0 9 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:10.848+0000 7fac606e4640 10 osd.0 9 handle_osd_map got inc map for epoch 10 2026-02-11T23:30:10.848+0000 7fac606e4640 20 osd.0 9 got_full_map 10, nothing requested 2026-02-11T23:30:10.848+0000 7fac606e4640 20 osd.0 9 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:10.848+0000 7fac606e4640 20 osd.0 9 handle_osd_map pg_num_history pg_num_history(e10 pg_nums {} deleted_pools ) 2026-02-11T23:30:10.848+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {10={}} 2026-02-11T23:30:10.848+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:10.848+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e10 maps [1~10] lci=[5,10] tlb=1) 2026-02-11T23:30:10.849+0000 7fac5e983640 10 osd.0 9 _committed_osd_maps 10..10 2026-02-11T23:30:10.849+0000 7fac5e983640 10 osd.0 9 advance to epoch 10 (<= last 10 <= newest_map 10) 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0 10 consume_map version 10 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0 9 await_reserved_maps epoch:10 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0 9 await_reserved_maps done 0 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0 10 consume_map 10 -- publish done 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:10.849+0000 7fac5e983640 20 osd.0 10 prune_pg_created 2026-02-11T23:30:10.849+0000 7fac5e983640 10 osd.0:0.consume_map 10 (was 9) 2026-02-11T23:30:10.849+0000 7fac5e983640 10 osd.0:1.consume_map 10 (was 9) 2026-02-11T23:30:10.849+0000 7fac5e983640 10 osd.0:2.consume_map 10 (was 9) 2026-02-11T23:30:10.849+0000 7fac5e983640 10 osd.0:3.consume_map 10 (was 9) 2026-02-11T23:30:10.849+0000 7fac5e983640 10 osd.0:4.consume_map 10 (was 9) 2026-02-11T23:30:10.849+0000 7fac5e983640 10 osd.0:5.consume_map 10 (was 9) 2026-02-11T23:30:10.849+0000 7fac5e983640 10 osd.0:6.consume_map 10 (was 9) 2026-02-11T23:30:10.849+0000 7fac5e983640 10 osd.0:7.consume_map 10 (was 9) 2026-02-11T23:30:10.849+0000 7fac5e983640 10 osd.0 10 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:10.849+0000 7fac5e983640 7 osd.0 10 activate_map version 10 2026-02-11T23:30:10.849+0000 7fac55971640 20 osd.0 10 agent_entry empty queue 2026-02-11T23:30:11.112+0000 7fac68beb640 10 osd.0 10 tick 2026-02-11T23:30:11.112+0000 7fac68beb640 20 osd.0 10 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:11.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:11.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:11.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:41.167614+0000) 2026-02-11T23:30:11.199+0000 7fac683ea640 10 osd.0 10 tick_without_osd_lock 2026-02-11T23:30:11.199+0000 7fac683ea640 0 osd.0 10 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:11.199+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:11.199+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:11.199+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:11.199+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.196 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:11.199+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:11.200, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:11.199+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:11.199+0000 7fac683ea640 10 osd.0 10 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:11.199+0000 7fac683ea640 20 osd.0 10 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:11.199+0000 7fac683ea640 10 osd.0 10 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:11.199+0000 7fac683ea640 20 osd.0 10 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:11.940+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 23 ==== osd_map(11..12 src has 1..12) ==== 1822+0+0 (crc 0 0 0) 0x38e2000 con 0x396bc00 2026-02-11T23:30:11.940+0000 7fac626e8640 20 osd.0 10 OSD::ms_dispatch: osd_map(11..12 src has 1..12) 2026-02-11T23:30:11.940+0000 7fac626e8640 20 osd.0 10 _dispatch 0x38e2000 osd_map(11..12 src has 1..12) 2026-02-11T23:30:11.940+0000 7fac626e8640 3 osd.0 10 handle_osd_map epochs [11,12], i have 10, src has [1,12] 2026-02-11T23:30:11.940+0000 7fac626e8640 10 osd.0 10 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:11.940+0000 7fac626e8640 10 osd.0 10 handle_osd_map got inc map for epoch 11 2026-02-11T23:30:11.940+0000 7fac626e8640 20 osd.0 10 got_full_map 11, nothing requested 2026-02-11T23:30:11.940+0000 7fac626e8640 10 osd.0 10 handle_osd_map got inc map for epoch 12 2026-02-11T23:30:11.940+0000 7fac626e8640 20 osd.0 10 got_full_map 12, nothing requested 2026-02-11T23:30:11.940+0000 7fac626e8640 20 osd.0 10 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:11.940+0000 7fac626e8640 20 osd.0 10 handle_osd_map pg_num_history pg_num_history(e12 pg_nums {} deleted_pools ) 2026-02-11T23:30:11.940+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {11={},12={}} 2026-02-11T23:30:11.940+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:11.940+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e12 maps [1~12] lci=[5,12] tlb=1) 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0 10 _committed_osd_maps 11..12 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0 10 advance to epoch 11 (<= last 12 <= newest_map 12) 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0 11 advance to epoch 12 (<= last 12 <= newest_map 12) 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0 12 consume_map version 12 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0 10 await_reserved_maps epoch:12 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0 10 await_reserved_maps done 0 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0 12 consume_map 12 -- publish done 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:11.940+0000 7fac5e983640 20 osd.0 12 prune_pg_created 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0:0.consume_map 12 (was 10) 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0:1.consume_map 12 (was 10) 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0:2.consume_map 12 (was 10) 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0:3.consume_map 12 (was 10) 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0:4.consume_map 12 (was 10) 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0:5.consume_map 12 (was 10) 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0:6.consume_map 12 (was 10) 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0:7.consume_map 12 (was 10) 2026-02-11T23:30:11.940+0000 7fac5e983640 10 osd.0 12 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:11.940+0000 7fac5e983640 7 osd.0 12 activate_map version 12 2026-02-11T23:30:11.940+0000 7fac55971640 20 osd.0 12 agent_entry empty queue 2026-02-11T23:30:12.067+0000 7fac68beb640 10 osd.0 12 tick 2026-02-11T23:30:12.067+0000 7fac68beb640 20 osd.0 12 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:12.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:12.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:12.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:42.167674+0000) 2026-02-11T23:30:12.179+0000 7fac683ea640 10 osd.0 12 tick_without_osd_lock 2026-02-11T23:30:12.179+0000 7fac683ea640 0 osd.0 12 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:12.179+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:12.179+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:12.179+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:12.179+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.196 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:12.179+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:12.179, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:12.179+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:12.179+0000 7fac683ea640 10 osd.0 12 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:12.179+0000 7fac683ea640 20 osd.0 12 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:12.179+0000 7fac683ea640 10 osd.0 12 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:12.179+0000 7fac683ea640 20 osd.0 12 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:12.375+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 24 ==== osd_map(12..13 src has 1..13) ==== 1563+0+0 (crc 0 0 0) 0x38e3180 con 0x396bc00 2026-02-11T23:30:12.375+0000 7fac626e8640 20 osd.0 12 OSD::ms_dispatch: osd_map(12..13 src has 1..13) 2026-02-11T23:30:12.375+0000 7fac626e8640 20 osd.0 12 _dispatch 0x38e3180 osd_map(12..13 src has 1..13) 2026-02-11T23:30:12.375+0000 7fac626e8640 3 osd.0 12 handle_osd_map epochs [12,13], i have 12, src has [1,13] 2026-02-11T23:30:12.375+0000 7fac626e8640 10 osd.0 12 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:12.375+0000 7fac626e8640 10 osd.0 12 handle_osd_map got inc map for epoch 13 2026-02-11T23:30:12.375+0000 7fac626e8640 20 osd.0 12 got_full_map 13, nothing requested 2026-02-11T23:30:12.375+0000 7fac626e8640 20 osd.0 12 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:12.375+0000 7fac626e8640 20 osd.0 12 handle_osd_map pg_num_history pg_num_history(e13 pg_nums {} deleted_pools ) 2026-02-11T23:30:12.375+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {13={}} 2026-02-11T23:30:12.375+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:12.375+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e13 maps [1~13] lci=[5,13] tlb=1) 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0 12 _committed_osd_maps 13..13 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0 12 advance to epoch 13 (<= last 13 <= newest_map 13) 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0 13 consume_map version 13 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0 12 await_reserved_maps epoch:13 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0 12 await_reserved_maps done 0 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0 13 consume_map 13 -- publish done 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0 13 prune_pg_created 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0:0.consume_map 13 (was 12) 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0:1.consume_map 13 (was 12) 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0:2.consume_map 13 (was 12) 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0:3.consume_map 13 (was 12) 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0:4.consume_map 13 (was 12) 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0:5.consume_map 13 (was 12) 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0:6.consume_map 13 (was 12) 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0:7.consume_map 13 (was 12) 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0 13 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0 13 maybe_update_heartbeat_peers updating 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0 13 adding neighbor peer osd.1 2026-02-11T23:30:12.375+0000 7fac5e983640 10 osd.0 13 adding neighbor peer osd.2 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0 13 get_con_osd_hb to osd.2 from_epoch 13 2026-02-11T23:30:12.375+0000 7fac5e983640 20 osd.0 13 get_nextmap_reserved map_reservations: {13=1} 2026-02-11T23:30:12.375+0000 7fac5e983640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] conn(0x396b800 0x3edab00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:12.375+0000 7fac5e983640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] conn(0x34c1000 0x3edb600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:12.376+0000 7fac5e983640 20 osd.0 13 release_map epoch: 13 2026-02-11T23:30:12.376+0000 7fac5e983640 10 osd.0 13 _add_heartbeat_peer: new peer osd.2 v2:127.0.0.1:6824/2662569638 v2:127.0.0.1:6822/2662569638 2026-02-11T23:30:12.376+0000 7fac5e983640 10 osd.0 13 maybe_update_heartbeat_peers 2 peers, extras 1,2 2026-02-11T23:30:12.376+0000 7fac5e983640 7 osd.0 13 activate_map version 13 2026-02-11T23:30:12.376+0000 7fac55971640 20 osd.0 13 agent_entry empty queue 2026-02-11T23:30:12.376+0000 7fac723fe640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] conn(0x396b800 0x3edab00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:12.376+0000 7fac723fe640 10 monclient: get_auth_request con 0x396b800 auth_method 0 2026-02-11T23:30:12.376+0000 7fac71bfd640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] conn(0x34c1000 0x3edb600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:12.376+0000 7fac71bfd640 10 monclient: get_auth_request con 0x34c1000 auth_method 0 2026-02-11T23:30:12.376+0000 7fac723fe640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] conn(0x396b800 0x3edab00 crc :-1 s=READY pgs=1 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:12.376+0000 7fac71bfd640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] conn(0x34c1000 0x3edb600 crc :-1 s=READY pgs=1 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:12.376+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> conn(0x3f56000 0x3edc100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:12.376+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> conn(0x3f56400 0x3edcc00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:12.376+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> conn(0x3f56000 0x3edc100 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:12.376+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> conn(0x3f56400 0x3edcc00 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:12.376+0000 7fac703fa640 10 osd.0 13 ms_handle_fast_authentication new session 0x38e3dc0 con 0x3f56400 entity osd.2 addr 2026-02-11T23:30:12.376+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> 127.0.0.1:0/2662569638 conn(0x3f56400 0x3edcc00 crc :-1 s=READY pgs=1 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:12.383+0000 7fac723fe640 10 osd.0 13 ms_handle_fast_authentication new session 0x38e2c40 con 0x3f56000 entity osd.2 addr 2026-02-11T23:30:12.383+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> 127.0.0.1:0/2662569638 conn(0x3f56000 0x3edc100 crc :-1 s=READY pgs=1 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:13.009+0000 7fac60ee5640 20 osd.0 13 reports for 0 queries 2026-02-11T23:30:13.009+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x34c3c00 con 0x396a800 2026-02-11T23:30:13.009+0000 7fac60ee5640 15 osd.0 13 collect_pg_stats 2026-02-11T23:30:13.009+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836482 v 0) -- 0x3f4c000 con 0x396a800 2026-02-11T23:30:13.090+0000 7fac68beb640 10 osd.0 13 tick 2026-02-11T23:30:13.090+0000 7fac68beb640 20 osd.0 13 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:13.165+0000 7fac683ea640 10 osd.0 13 tick_without_osd_lock 2026-02-11T23:30:13.165+0000 7fac683ea640 0 osd.0 13 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:13.165+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:13.165+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:13.165+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:13.165+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.196 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:13.165+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:13.165, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:13.165+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:13.165+0000 7fac683ea640 10 osd.0 13 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:13.165+0000 7fac683ea640 20 osd.0 13 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:13.165+0000 7fac683ea640 10 osd.0 13 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:13.165+0000 7fac683ea640 20 osd.0 13 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:13.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:13.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:13.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:43.167718+0000) 2026-02-11T23:30:14.086+0000 7fac68beb640 10 osd.0 13 tick 2026-02-11T23:30:14.086+0000 7fac68beb640 20 osd.0 13 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:14.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:14.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:14.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:44.167776+0000) 2026-02-11T23:30:14.193+0000 7fac683ea640 10 osd.0 13 tick_without_osd_lock 2026-02-11T23:30:14.193+0000 7fac683ea640 0 osd.0 13 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:14.193+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:14.193+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.196 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:14.193+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:14.193, recovery is active?:false restrictions:<....> 2026-02-11T23:30:14.193+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:14.193+0000 7fac683ea640 10 osd.0 13 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:14.193+0000 7fac683ea640 20 osd.0 13 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:14.193+0000 7fac683ea640 10 osd.0 13 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:14.193+0000 7fac683ea640 20 osd.0 13 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:15.035+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> conn(0x3f56800 0x3edd180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:15.035+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> conn(0x3f56800 0x3edd180 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:15.035+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> conn(0x3f56c00 0x3edd700 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:15.035+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> conn(0x3f56c00 0x3edd700 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:15.043+0000 7fac68beb640 10 osd.0 13 tick 2026-02-11T23:30:15.043+0000 7fac68beb640 20 osd.0 13 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:15.044+0000 7fac723fe640 10 osd.0 13 ms_handle_fast_authentication new session 0x38e2a80 con 0x3f56c00 entity osd.3 addr 2026-02-11T23:30:15.044+0000 7fac71bfd640 10 osd.0 13 ms_handle_fast_authentication new session 0x34bfc00 con 0x3f56800 entity osd.3 addr 2026-02-11T23:30:15.044+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> 127.0.0.1:0/1114810440 conn(0x3f56800 0x3edd180 crc :-1 s=READY pgs=1 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:15.044+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> 127.0.0.1:0/1114810440 conn(0x3f56c00 0x3edd700 crc :-1 s=READY pgs=1 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:15.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:15.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:15.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:45.167798+0000) 2026-02-11T23:30:15.220+0000 7fac683ea640 10 osd.0 13 tick_without_osd_lock 2026-02-11T23:30:15.220+0000 7fac683ea640 0 osd.0 13 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:15.220+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:15.220+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.180 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:15.220+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:15.220, recovery is active?:false restrictions:<....> 2026-02-11T23:30:15.220+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:15.220+0000 7fac683ea640 10 osd.0 13 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:15.220+0000 7fac683ea640 20 osd.0 13 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:15.220+0000 7fac683ea640 10 osd.0 13 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:15.220+0000 7fac683ea640 20 osd.0 13 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:15.268+0000 7fac4d160640 5 osd.0 13 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x69e7/0x1c000, compress 0x0/0x0/0x0, omap 0x1ab1, meta 0x1a4e54f), peers [1,2] op hist []) 2026-02-11T23:30:15.268+0000 7fac4d160640 20 osd.0 13 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:15.268+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e13 up_from 5 ping_stamp 2026-02-11T23:30:15.268882+0000/8.103298187s send_stamp 8.103298187s delta_ub 1.980984926s) -- 0x3b1a600 con 0x396b400 2026-02-11T23:30:15.268+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e13 up_from 5 ping_stamp 2026-02-11T23:30:15.268882+0000/8.103298187s send_stamp 8.103298187s delta_ub 1.980984926s) -- 0x3b1a000 con 0x3752400 2026-02-11T23:30:15.268+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e13 up_from 5 ping_stamp 2026-02-11T23:30:15.268882+0000/8.103298187s send_stamp 8.103298187s) -- 0x3b1a900 con 0x396b800 2026-02-11T23:30:15.268+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e13 up_from 5 ping_stamp 2026-02-11T23:30:15.268882+0000/8.103298187s send_stamp 8.103298187s) -- 0x3b1aa80 con 0x34c1000 2026-02-11T23:30:15.268+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 1 ==== osd_ping(ping_reply e17 up_from 13 ping_stamp 2026-02-11T23:30:15.268882+0000/8.103298187s send_stamp 3.454393864s delta_ub -4.648904800s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a900 con 0x396b800 2026-02-11T23:30:15.268+0000 7fac723fe640 20 osd.0 13 get_con_osd_cluster to osd.2 from_epoch 13 2026-02-11T23:30:15.268+0000 7fac723fe640 20 osd.0 13 get_nextmap_reserved map_reservations: {13=1} 2026-02-11T23:30:15.268+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] conn(0x3f57c00 0x38a6100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:15.268+0000 7fac723fe640 20 osd.0 13 release_map epoch: 13 2026-02-11T23:30:15.268+0000 7fac723fe640 20 osd.0 13 handle_osd_ping new stamps hbstamp(osd.2 up_from 0 peer_clock_delta [-4.649084568s,-4.648904800s]) 2026-02-11T23:30:15.268+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> conn(0x3f57000 0x3e36000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:15.268+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> conn(0x3f57000 0x3e36000 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:15.268+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 1 ==== osd_ping(ping_reply e17 up_from 13 ping_stamp 2026-02-11T23:30:15.268882+0000/8.103298187s send_stamp 3.454423666s delta_ub -4.648875237s) ==== 2033+0+0 (crc 0 0 0) 0x3b1aa80 con 0x34c1000 2026-02-11T23:30:15.268+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 2 ==== osd_ping(ping_reply e14 up_from 9 ping_stamp 2026-02-11T23:30:15.268882+0000/8.103298187s send_stamp 6.122543335s delta_ub -1.980755448s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a480 con 0x396b400 2026-02-11T23:30:15.268+0000 7fac71bfd640 20 osd.0 13 get_con_osd_cluster to osd.2 from_epoch 13 2026-02-11T23:30:15.268+0000 7fac71bfd640 20 osd.0 13 get_nextmap_reserved map_reservations: {13=1} 2026-02-11T23:30:15.268+0000 7fac71bfd640 20 osd.0 13 release_map epoch: 13 2026-02-11T23:30:15.268+0000 7fac71bfd640 20 osd.0 13 handle_osd_ping new stamps hbstamp(osd.2 up_from 0 peer_clock_delta [-4.649194717s,-4.648875237s]) 2026-02-11T23:30:15.268+0000 7fac703fa640 20 osd.0 13 get_con_osd_cluster to osd.1 from_epoch 13 2026-02-11T23:30:15.268+0000 7fac703fa640 20 osd.0 13 get_nextmap_reserved map_reservations: {13=1} 2026-02-11T23:30:15.268+0000 7fac703fa640 20 osd.0 13 release_map epoch: 13 2026-02-11T23:30:15.268+0000 7fac703fa640 10 osd.0 13 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 10 to ping map epoch of 14 2026-02-11T23:30:15.268+0000 7fac703fa640 10 osd.0 13 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 13 is not newer than session's projected_epoch of 14 2026-02-11T23:30:15.268+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 2 ==== osd_ping(ping_reply e14 up_from 9 ping_stamp 2026-02-11T23:30:15.268882+0000/8.103298187s send_stamp 6.122719288s delta_ub -1.980578899s) ==== 2033+0+0 (crc 0 0 0) 0x3b1aa80 con 0x3752400 2026-02-11T23:30:15.268+0000 7fac703fa640 20 osd.0 13 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.981104493s,-1.980755448s]) 2026-02-11T23:30:15.268+0000 7fac71bfd640 20 osd.0 13 get_con_osd_cluster to osd.1 from_epoch 13 2026-02-11T23:30:15.268+0000 7fac71bfd640 20 osd.0 13 get_nextmap_reserved map_reservations: {13=1} 2026-02-11T23:30:15.268+0000 7fac71bfd640 20 osd.0 13 release_map epoch: 13 2026-02-11T23:30:15.268+0000 7fac71bfd640 10 osd.0 13 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 13 is not newer than session's projected_epoch of 14 2026-02-11T23:30:15.268+0000 7fac71bfd640 20 osd.0 13 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980948687s,-1.980578899s]) 2026-02-11T23:30:15.268+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] conn(0x3f57c00 0x38a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:15.268+0000 7fac71bfd640 10 osd.0 13 ms_handle_fast_authentication new session 0x3940700 con 0x3f57000 entity osd.2 addr 2026-02-11T23:30:15.268+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 2 ==== osd_map(14..14 src has 1..14) ==== 324+0+0 (unknown 0 0 0) 0x3940000 con 0x3752000 2026-02-11T23:30:15.268+0000 7fac606e4640 20 osd.0 13 OSD::ms_dispatch: osd_map(14..14 src has 1..14) 2026-02-11T23:30:15.268+0000 7fac606e4640 20 osd.0 13 _dispatch 0x3940000 osd_map(14..14 src has 1..14) 2026-02-11T23:30:15.268+0000 7fac703fa640 10 monclient: get_auth_request con 0x3f57c00 auth_method 0 2026-02-11T23:30:15.268+0000 7fac606e4640 3 osd.0 13 handle_osd_map epochs [14,14], i have 13, src has [1,14] 2026-02-11T23:30:15.268+0000 7fac606e4640 10 osd.0 13 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:15.268+0000 7fac606e4640 10 osd.0 13 handle_osd_map got inc map for epoch 14 2026-02-11T23:30:15.268+0000 7fac606e4640 20 osd.0 13 got_full_map 14, nothing requested 2026-02-11T23:30:15.268+0000 7fac606e4640 20 osd.0 13 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:15.268+0000 7fac606e4640 20 osd.0 13 handle_osd_map pg_num_history pg_num_history(e14 pg_nums {} deleted_pools ) 2026-02-11T23:30:15.268+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {14={}} 2026-02-11T23:30:15.268+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:15.268+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e14 maps [1~14] lci=[5,14] tlb=1) 2026-02-11T23:30:15.268+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] conn(0x3f57000 0x3e36000 crc :-1 s=SESSION_ACCEPTING pgs=1 gs=0 cs=0 l=0 c_cookie=1921139330285901102 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_existing_connection connection race detected, replacing existing=0x3f57c00 socket by this connection's socket 2026-02-11T23:30:15.269+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] conn(0x3f57000 0x3e36000 crc :-1 s=SESSION_ACCEPTING pgs=1 gs=0 cs=0 l=0 c_cookie=1921139330285901102 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:15.269+0000 7fac71bfd640 10 osd.0 13 new session (incoming)0x39408c0 con=0x3f57c00 addr=v2:127.0.0.1:6820/2662569638 must have raced with connect 2026-02-11T23:30:15.269+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] conn(0x3f57c00 0x38a6100 unknown :-1 s=READY pgs=1 gs=4 cs=0 l=0 c_cookie=1921139330285901102 s_cookie=16764330780886106243 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=1aa9420ad1b9152e server_cookie=e8a6de0623dd0083 in_seq=0 out_seq=0 2026-02-11T23:30:15.269+0000 7fac606e4640 2 osd.0 13 ms_handle_reset con 0x3f57000 session 0x3940700 2026-02-11T23:30:15.269+0000 7fac5e983640 10 osd.0 13 _committed_osd_maps 14..14 2026-02-11T23:30:15.269+0000 7fac5e983640 10 osd.0 13 advance to epoch 14 (<= last 14 <= newest_map 14) 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0 14 consume_map version 14 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0 13 await_reserved_maps epoch:14 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0 13 await_reserved_maps done 0 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0 14 consume_map 14 -- publish done 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:15.269+0000 7fac5e983640 20 osd.0 14 prune_pg_created 2026-02-11T23:30:15.269+0000 7fac5e983640 10 osd.0:0.consume_map 14 (was 13) 2026-02-11T23:30:15.269+0000 7fac5e983640 10 osd.0:1.consume_map 14 (was 13) 2026-02-11T23:30:15.269+0000 7fac5e983640 10 osd.0:2.consume_map 14 (was 13) 2026-02-11T23:30:15.269+0000 7fac5e983640 10 osd.0:3.consume_map 14 (was 13) 2026-02-11T23:30:15.269+0000 7fac5e983640 10 osd.0:4.consume_map 14 (was 13) 2026-02-11T23:30:15.269+0000 7fac5e983640 10 osd.0:5.consume_map 14 (was 13) 2026-02-11T23:30:15.269+0000 7fac5e983640 10 osd.0:6.consume_map 14 (was 13) 2026-02-11T23:30:15.269+0000 7fac5e983640 10 osd.0:7.consume_map 14 (was 13) 2026-02-11T23:30:15.269+0000 7fac5e983640 10 osd.0 14 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:15.269+0000 7fac5e983640 7 osd.0 14 activate_map version 14 2026-02-11T23:30:15.269+0000 7fac55971640 20 osd.0 14 agent_entry empty queue 2026-02-11T23:30:16.089+0000 7fac68beb640 10 osd.0 14 tick 2026-02-11T23:30:16.089+0000 7fac68beb640 20 osd.0 14 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:16.148+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 2 ==== osd_ping(ping e14 up_from 9 ping_stamp 2026-02-11T23:30:16.149056+0000/7.002620697s send_stamp 7.002620697s delta_ub -1.980578899s) ==== 2033+0+0 (crc 0 0 0) 0x3b1aa80 con 0x396a400 2026-02-11T23:30:16.148+0000 7fac71bfd640 20 osd.0 14 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980962276s,-1.980578899s]) 2026-02-11T23:30:16.148+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e14 up_from 5 ping_stamp 2026-02-11T23:30:16.149056+0000/7.002620697s send_stamp 8.983582497s delta_ub 1.980962276s) -- 0x3b1a000 con 0x396a400 2026-02-11T23:30:16.148+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 2 ==== osd_ping(ping e14 up_from 9 ping_stamp 2026-02-11T23:30:16.149056+0000/7.002620697s send_stamp 7.002620697s delta_ub -1.980578899s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a900 con 0x396a000 2026-02-11T23:30:16.148+0000 7fac71bfd640 20 osd.0 14 get_con_osd_cluster to osd.1 from_epoch 14 2026-02-11T23:30:16.148+0000 7fac71bfd640 20 osd.0 14 get_nextmap_reserved map_reservations: {14=1} 2026-02-11T23:30:16.148+0000 7fac71bfd640 20 osd.0 14 release_map epoch: 14 2026-02-11T23:30:16.148+0000 7fac71bfd640 10 osd.0 14 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 14 is not newer than session's projected_epoch of 14 2026-02-11T23:30:16.148+0000 7fac723fe640 20 osd.0 14 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980987549s,-1.980578899s]) 2026-02-11T23:30:16.148+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e14 up_from 5 ping_stamp 2026-02-11T23:30:16.149056+0000/7.002620697s send_stamp 8.983608246s delta_ub 1.980987549s) -- 0x3919980 con 0x396a000 2026-02-11T23:30:16.148+0000 7fac723fe640 20 osd.0 14 get_con_osd_cluster to osd.1 from_epoch 14 2026-02-11T23:30:16.148+0000 7fac723fe640 20 osd.0 14 get_nextmap_reserved map_reservations: {14=1} 2026-02-11T23:30:16.148+0000 7fac723fe640 20 osd.0 14 release_map epoch: 14 2026-02-11T23:30:16.148+0000 7fac723fe640 10 osd.0 14 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 14 is not newer than session's projected_epoch of 14 2026-02-11T23:30:16.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:16.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:16.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:46.167864+0000) 2026-02-11T23:30:16.200+0000 7fac683ea640 10 osd.0 14 tick_without_osd_lock 2026-02-11T23:30:16.200+0000 7fac683ea640 0 osd.0 14 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:16.200+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:16.200+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.180 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:16.200+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:16.200, recovery is active?:false restrictions:<....> 2026-02-11T23:30:16.200+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:16.200+0000 7fac683ea640 10 osd.0 14 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:16.200+0000 7fac683ea640 20 osd.0 14 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:16.200+0000 7fac683ea640 10 osd.0 14 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:16.200+0000 7fac683ea640 20 osd.0 14 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:16.368+0000 7fac4d160640 5 osd.0 14 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x73b9/0x1e000, compress 0x0/0x0/0x0, omap 0x1ab1, meta 0x1a4e54f), peers [1,2] op hist []) 2026-02-11T23:30:16.368+0000 7fac4d160640 20 osd.0 14 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:16.368+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e14 up_from 5 ping_stamp 2026-02-11T23:30:16.369032+0000/9.203447342s send_stamp 9.203447342s delta_ub 1.980987549s) -- 0x3b1ac00 con 0x396b400 2026-02-11T23:30:16.368+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e14 up_from 5 ping_stamp 2026-02-11T23:30:16.369032+0000/9.203447342s send_stamp 9.203447342s delta_ub 1.980987549s) -- 0x3b1ad80 con 0x3752400 2026-02-11T23:30:16.368+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e14 up_from 5 ping_stamp 2026-02-11T23:30:16.369032+0000/9.203447342s send_stamp 9.203447342s delta_ub 4.649194717s) -- 0x3b1af00 con 0x396b800 2026-02-11T23:30:16.368+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e14 up_from 5 ping_stamp 2026-02-11T23:30:16.369032+0000/9.203447342s send_stamp 9.203447342s delta_ub 4.649194717s) -- 0x3b1b080 con 0x34c1000 2026-02-11T23:30:16.368+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 3 ==== osd_ping(ping_reply e14 up_from 9 ping_stamp 2026-02-11T23:30:16.369032+0000/9.203447342s send_stamp 7.222702980s delta_ub -1.980744720s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ac00 con 0x396b400 2026-02-11T23:30:16.368+0000 7fac703fa640 20 osd.0 14 get_con_osd_cluster to osd.1 from_epoch 14 2026-02-11T23:30:16.368+0000 7fac703fa640 20 osd.0 14 get_nextmap_reserved map_reservations: {14=1} 2026-02-11T23:30:16.368+0000 7fac703fa640 20 osd.0 14 release_map epoch: 14 2026-02-11T23:30:16.368+0000 7fac703fa640 10 osd.0 14 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 14 is not newer than session's projected_epoch of 14 2026-02-11T23:30:16.368+0000 7fac703fa640 20 osd.0 14 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980911136s,-1.980744720s]) 2026-02-11T23:30:16.368+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 3 ==== osd_ping(ping_reply e14 up_from 9 ping_stamp 2026-02-11T23:30:16.369032+0000/9.203447342s send_stamp 7.222723007s delta_ub -1.980724812s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b080 con 0x3752400 2026-02-11T23:30:16.368+0000 7fac71bfd640 20 osd.0 14 get_con_osd_cluster to osd.1 from_epoch 14 2026-02-11T23:30:16.368+0000 7fac71bfd640 20 osd.0 14 get_nextmap_reserved map_reservations: {14=1} 2026-02-11T23:30:16.368+0000 7fac71bfd640 20 osd.0 14 release_map epoch: 14 2026-02-11T23:30:16.368+0000 7fac71bfd640 10 osd.0 14 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 14 is not newer than session's projected_epoch of 14 2026-02-11T23:30:16.368+0000 7fac71bfd640 20 osd.0 14 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980902195s,-1.980724812s]) 2026-02-11T23:30:16.369+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 2 ==== osd_ping(ping_reply e17 up_from 13 ping_stamp 2026-02-11T23:30:16.369032+0000/9.203447342s send_stamp 4.554546833s delta_ub -4.648900986s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b080 con 0x34c1000 2026-02-11T23:30:16.369+0000 7fac71bfd640 20 osd.0 14 get_con_osd_cluster to osd.2 from_epoch 14 2026-02-11T23:30:16.369+0000 7fac71bfd640 20 osd.0 14 get_nextmap_reserved map_reservations: {14=1} 2026-02-11T23:30:16.369+0000 7fac71bfd640 20 osd.0 14 release_map epoch: 14 2026-02-11T23:30:16.369+0000 7fac71bfd640 10 osd.0 14 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 0 to ping map epoch of 17 2026-02-11T23:30:16.369+0000 7fac71bfd640 10 osd.0 14 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 14 is not newer than session's projected_epoch of 17 2026-02-11T23:30:16.369+0000 7fac71bfd640 20 osd.0 14 handle_osd_ping new stamps hbstamp(osd.2 up_from 0 peer_clock_delta [-4.649425983s,-4.648900986s]) 2026-02-11T23:30:16.369+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 2 ==== osd_ping(ping_reply e17 up_from 13 ping_stamp 2026-02-11T23:30:16.369032+0000/9.203447342s send_stamp 4.554924488s delta_ub -4.648523331s) ==== 2033+0+0 (crc 0 0 0) 0x3b1af00 con 0x396b800 2026-02-11T23:30:16.369+0000 7fac723fe640 20 osd.0 14 get_con_osd_cluster to osd.2 from_epoch 14 2026-02-11T23:30:16.369+0000 7fac723fe640 20 osd.0 14 get_nextmap_reserved map_reservations: {14=1} 2026-02-11T23:30:16.369+0000 7fac723fe640 20 osd.0 14 release_map epoch: 14 2026-02-11T23:30:16.369+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.2 v2:127.0.0.1:6820/2662569638 1 ==== osd_map(15..17 src has 1..17) ==== 2522+0+0 (unknown 0 0 0) 0x3940a80 con 0x3f57c00 2026-02-11T23:30:16.369+0000 7fac723fe640 10 osd.0 14 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 14 is not newer than session's projected_epoch of 17 2026-02-11T23:30:16.369+0000 7fac606e4640 20 osd.0 14 OSD::ms_dispatch: osd_map(15..17 src has 1..17) 2026-02-11T23:30:16.369+0000 7fac606e4640 20 osd.0 14 _dispatch 0x3940a80 osd_map(15..17 src has 1..17) 2026-02-11T23:30:16.369+0000 7fac723fe640 20 osd.0 14 handle_osd_ping new stamps hbstamp(osd.2 up_from 0 peer_clock_delta [-4.649074078s,-4.648523331s]) 2026-02-11T23:30:16.369+0000 7fac606e4640 3 osd.0 14 handle_osd_map epochs [15,17], i have 14, src has [1,17] 2026-02-11T23:30:16.369+0000 7fac606e4640 10 osd.0 14 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:16.369+0000 7fac606e4640 10 osd.0 14 handle_osd_map got inc map for epoch 15 2026-02-11T23:30:16.369+0000 7fac606e4640 20 osd.0 14 got_full_map 15, nothing requested 2026-02-11T23:30:16.369+0000 7fac606e4640 10 osd.0 14 handle_osd_map got inc map for epoch 16 2026-02-11T23:30:16.369+0000 7fac606e4640 20 osd.0 14 got_full_map 16, nothing requested 2026-02-11T23:30:16.369+0000 7fac606e4640 10 osd.0 14 handle_osd_map got inc map for epoch 17 2026-02-11T23:30:16.369+0000 7fac606e4640 20 osd.0 14 got_full_map 17, nothing requested 2026-02-11T23:30:16.369+0000 7fac606e4640 20 osd.0 14 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:16.369+0000 7fac606e4640 20 osd.0 14 handle_osd_map pg_num_history pg_num_history(e17 pg_nums {} deleted_pools ) 2026-02-11T23:30:16.369+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {15={},16={},17={}} 2026-02-11T23:30:16.369+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:16.369+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e17 maps [1~17] lci=[5,17] tlb=1) 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0 14 _committed_osd_maps 15..17 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0 14 advance to epoch 15 (<= last 17 <= newest_map 17) 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0 15 advance to epoch 16 (<= last 17 <= newest_map 17) 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0 16 advance to epoch 17 (<= last 17 <= newest_map 17) 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0 17 consume_map version 17 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0 14 await_reserved_maps epoch:17 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0 14 await_reserved_maps done 0 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0 17 consume_map 17 -- publish done 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0 17 prune_pg_created 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0:0.consume_map 17 (was 14) 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0:1.consume_map 17 (was 14) 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0:2.consume_map 17 (was 14) 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0:3.consume_map 17 (was 14) 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0:4.consume_map 17 (was 14) 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0:5.consume_map 17 (was 14) 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0:6.consume_map 17 (was 14) 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0:7.consume_map 17 (was 14) 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0 17 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0 17 maybe_update_heartbeat_peers updating 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0 17 adding neighbor peer osd.1 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0 17 adding neighbor peer osd.3 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0 17 get_con_osd_hb to osd.3 from_epoch 17 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0 17 get_nextmap_reserved map_reservations: {17=1} 2026-02-11T23:30:16.369+0000 7fac5e983640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] conn(0x3740000 0x3edbb80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:16.369+0000 7fac5e983640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] conn(0x3740400 0x38a5600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:16.369+0000 7fac5e983640 20 osd.0 17 release_map epoch: 17 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0 17 _add_heartbeat_peer: new peer osd.3 v2:127.0.0.1:6832/1114810440 v2:127.0.0.1:6830/1114810440 2026-02-11T23:30:16.369+0000 7fac5e983640 10 osd.0 17 maybe_update_heartbeat_peers 3 peers, extras 1,2,3 2026-02-11T23:30:16.369+0000 7fac5e983640 7 osd.0 17 activate_map version 17 2026-02-11T23:30:16.369+0000 7fac55971640 20 osd.0 17 agent_entry empty queue 2026-02-11T23:30:16.369+0000 7fac703fa640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] conn(0x3740000 0x3edbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:16.369+0000 7fac723fe640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] conn(0x3740400 0x38a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:16.369+0000 7fac723fe640 10 monclient: get_auth_request con 0x3740400 auth_method 0 2026-02-11T23:30:16.369+0000 7fac703fa640 10 monclient: get_auth_request con 0x3740000 auth_method 0 2026-02-11T23:30:16.369+0000 7fac723fe640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] conn(0x3740400 0x38a5600 crc :-1 s=READY pgs=2 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:16.369+0000 7fac703fa640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] conn(0x3740000 0x3edbb80 crc :-1 s=READY pgs=2 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:17.116+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 1 ==== osd_ping(ping e17 up_from 13 ping_stamp 2026-02-11T23:30:17.117301+0000/5.302697182s send_stamp 5.302697182s delta_ub -4.648523331s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ac00 con 0x3f56400 2026-02-11T23:30:17.116+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 1 ==== osd_ping(ping e17 up_from 13 ping_stamp 2026-02-11T23:30:17.117301+0000/5.302697182s send_stamp 5.302697182s delta_ub -4.648523331s) ==== 2033+0+0 (crc 0 0 0) 0x3b1af00 con 0x3f56000 2026-02-11T23:30:17.116+0000 7fac703fa640 20 osd.0 17 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649113178s,-4.648523331s]) 2026-02-11T23:30:17.116+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e17 up_from 5 ping_stamp 2026-02-11T23:30:17.117301+0000/5.302697182s send_stamp 9.951810837s delta_ub 4.649113178s) -- 0x3b1a480 con 0x3f56400 2026-02-11T23:30:17.117+0000 7fac703fa640 20 osd.0 17 get_con_osd_cluster to osd.2 from_epoch 17 2026-02-11T23:30:17.117+0000 7fac703fa640 20 osd.0 17 get_nextmap_reserved map_reservations: {17=1} 2026-02-11T23:30:17.117+0000 7fac703fa640 20 osd.0 17 release_map epoch: 17 2026-02-11T23:30:17.117+0000 7fac703fa640 10 osd.0 17 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 17 is not newer than session's projected_epoch of 17 2026-02-11T23:30:17.117+0000 7fac723fe640 20 osd.0 17 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649137020s,-4.648523331s]) 2026-02-11T23:30:17.117+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e17 up_from 5 ping_stamp 2026-02-11T23:30:17.117301+0000/5.302697182s send_stamp 9.951833725s delta_ub 4.649137020s) -- 0x3919980 con 0x3f56000 2026-02-11T23:30:17.117+0000 7fac723fe640 20 osd.0 17 get_con_osd_cluster to osd.2 from_epoch 17 2026-02-11T23:30:17.117+0000 7fac723fe640 20 osd.0 17 get_nextmap_reserved map_reservations: {17=1} 2026-02-11T23:30:17.117+0000 7fac723fe640 20 osd.0 17 release_map epoch: 17 2026-02-11T23:30:17.117+0000 7fac723fe640 10 osd.0 17 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 17 is not newer than session's projected_epoch of 17 2026-02-11T23:30:17.138+0000 7fac68beb640 10 osd.0 17 tick 2026-02-11T23:30:17.138+0000 7fac68beb640 20 osd.0 17 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:17.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:17.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:17.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:47.167911+0000) 2026-02-11T23:30:17.214+0000 7fac683ea640 10 osd.0 17 tick_without_osd_lock 2026-02-11T23:30:17.214+0000 7fac683ea640 0 osd.0 17 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:17.214+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:17.214+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.180 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:17.214+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:17.214, recovery is active?:false restrictions:<....> 2026-02-11T23:30:17.214+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:17.214+0000 7fac683ea640 10 osd.0 17 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:17.214+0000 7fac683ea640 20 osd.0 17 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:17.214+0000 7fac683ea640 10 osd.0 17 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:17.214+0000 7fac683ea640 20 osd.0 17 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:17.675+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 25 ==== osd_map(20..21 src has 1..21) ==== 1637+0+0 (crc 0 0 0) 0x38e2700 con 0x396bc00 2026-02-11T23:30:17.675+0000 7fac626e8640 20 osd.0 17 OSD::ms_dispatch: osd_map(20..21 src has 1..21) 2026-02-11T23:30:17.675+0000 7fac626e8640 20 osd.0 17 _dispatch 0x38e2700 osd_map(20..21 src has 1..21) 2026-02-11T23:30:17.675+0000 7fac626e8640 10 monclient: _renew_subs 2026-02-11T23:30:17.675+0000 7fac626e8640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:17.675+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=18}) -- 0x389ab60 con 0x396bc00 2026-02-11T23:30:17.675+0000 7fac626e8640 3 osd.0 17 handle_osd_map epochs [20,21], i have 17, src has [1,21] 2026-02-11T23:30:17.675+0000 7fac626e8640 10 osd.0 17 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:17.675+0000 7fac626e8640 10 osd.0 17 handle_osd_map message skips epochs 18..19 2026-02-11T23:30:17.675+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 26 ==== osd_map(18..21 src has 1..21) ==== 2884+0+0 (crc 0 0 0) 0x3940700 con 0x396bc00 2026-02-11T23:30:17.675+0000 7fac626e8640 20 osd.0 17 OSD::ms_dispatch: osd_map(18..21 src has 1..21) 2026-02-11T23:30:17.675+0000 7fac626e8640 20 osd.0 17 _dispatch 0x3940700 osd_map(18..21 src has 1..21) 2026-02-11T23:30:17.676+0000 7fac626e8640 3 osd.0 17 handle_osd_map epochs [18,21], i have 17, src has [1,21] 2026-02-11T23:30:17.676+0000 7fac626e8640 10 osd.0 17 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:17.676+0000 7fac626e8640 10 osd.0 17 handle_osd_map got inc map for epoch 18 2026-02-11T23:30:17.676+0000 7fac626e8640 20 osd.0 17 got_full_map 18, nothing requested 2026-02-11T23:30:17.676+0000 7fac626e8640 10 osd.0 17 handle_osd_map got inc map for epoch 19 2026-02-11T23:30:17.676+0000 7fac626e8640 20 osd.0 17 got_full_map 19, nothing requested 2026-02-11T23:30:17.676+0000 7fac626e8640 10 osd.0 17 handle_osd_map got inc map for epoch 20 2026-02-11T23:30:17.676+0000 7fac626e8640 20 osd.0 17 got_full_map 20, nothing requested 2026-02-11T23:30:17.676+0000 7fac626e8640 10 osd.0 17 handle_osd_map got inc map for epoch 21 2026-02-11T23:30:17.676+0000 7fac626e8640 20 osd.0 17 got_full_map 21, nothing requested 2026-02-11T23:30:17.676+0000 7fac626e8640 20 osd.0 17 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:17.676+0000 7fac626e8640 20 osd.0 17 handle_osd_map pg_num_history pg_num_history(e21 pg_nums {} deleted_pools ) 2026-02-11T23:30:17.676+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {18={},19={},20={},21={}} 2026-02-11T23:30:17.676+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:17.676+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e21 maps [1~21] lci=[5,21] tlb=1) 2026-02-11T23:30:17.676+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> conn(0x3f57400 0x3e36580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:17.676+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> conn(0x3f57800 0x3e36b00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:17.676+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> conn(0x3f57400 0x3e36580 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:17.676+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> conn(0x3f57800 0x3e36b00 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0 17 _committed_osd_maps 18..21 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0 17 advance to epoch 18 (<= last 21 <= newest_map 21) 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0 18 advance to epoch 19 (<= last 21 <= newest_map 21) 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0 19 advance to epoch 20 (<= last 21 <= newest_map 21) 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0 20 advance to epoch 21 (<= last 21 <= newest_map 21) 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0 21 consume_map version 21 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0 17 await_reserved_maps epoch:21 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0 17 await_reserved_maps done 0 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0 21 consume_map 21 -- publish done 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0 21 prune_pg_created 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0:0.consume_map 21 (was 17) 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0:1.consume_map 21 (was 17) 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0:2.consume_map 21 (was 17) 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0:3.consume_map 21 (was 17) 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0:4.consume_map 21 (was 17) 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0:5.consume_map 21 (was 17) 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0:6.consume_map 21 (was 17) 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0:7.consume_map 21 (was 17) 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0 21 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0 21 maybe_update_heartbeat_peers updating 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0 21 adding neighbor peer osd.1 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0 21 adding neighbor peer osd.4 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0 21 get_con_osd_hb to osd.4 from_epoch 21 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:17.676+0000 7fac5e983640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] conn(0x3740800 0x3e38680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:17.676+0000 7fac5e983640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] conn(0x3740c00 0x3e38c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:17.676+0000 7fac5e983640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0 21 _add_heartbeat_peer: new peer osd.4 v2:127.0.0.1:6840/864727417 v2:127.0.0.1:6838/864727417 2026-02-11T23:30:17.676+0000 7fac5e983640 10 osd.0 21 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:30:17.676+0000 7fac5e983640 7 osd.0 21 activate_map version 21 2026-02-11T23:30:17.676+0000 7fac55971640 20 osd.0 21 agent_entry empty queue 2026-02-11T23:30:17.684+0000 7fac703fa640 10 osd.0 21 ms_handle_fast_authentication new session 0x39401c0 con 0x3f57400 entity osd.4 addr 2026-02-11T23:30:17.684+0000 7fac71bfd640 10 osd.0 21 ms_handle_fast_authentication new session 0x3941340 con 0x3f57800 entity osd.4 addr 2026-02-11T23:30:17.684+0000 7fac723fe640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] conn(0x3740800 0x3e38680 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:17.684+0000 7fac723fe640 10 monclient: get_auth_request con 0x3740800 auth_method 0 2026-02-11T23:30:17.684+0000 7fac703fa640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] conn(0x3740c00 0x3e38c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:17.684+0000 7fac703fa640 10 monclient: get_auth_request con 0x3740c00 auth_method 0 2026-02-11T23:30:17.684+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] >> 127.0.0.1:0/864727417 conn(0x3f57400 0x3e36580 crc :-1 s=READY pgs=1 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:17.684+0000 7fac723fe640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] conn(0x3740800 0x3e38680 crc :-1 s=READY pgs=1 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:17.684+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] >> 127.0.0.1:0/864727417 conn(0x3f57800 0x3e36b00 crc :-1 s=READY pgs=2 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:17.684+0000 7fac703fa640 1 --2- 127.0.0.1:0/460666604 >> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] conn(0x3740c00 0x3e38c00 crc :-1 s=READY pgs=1 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:17.849+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 3 ==== osd_ping(ping e20 up_from 9 ping_stamp 2026-02-11T23:30:17.849238+0000/8.702802658s send_stamp 8.702802658s delta_ub -1.980724812s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b080 con 0x396a400 2026-02-11T23:30:17.849+0000 7fac71bfd640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980941653s,-1.980724812s]) 2026-02-11T23:30:17.849+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e21 up_from 5 ping_stamp 2026-02-11T23:30:17.849238+0000/8.702802658s send_stamp 10.683744431s delta_ub 1.980941653s) -- 0x3b1a600 con 0x396a400 2026-02-11T23:30:17.849+0000 7fac71bfd640 20 osd.0 21 get_con_osd_cluster to osd.1 from_epoch 21 2026-02-11T23:30:17.849+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 3 ==== osd_ping(ping e20 up_from 9 ping_stamp 2026-02-11T23:30:17.849238+0000/8.702802658s send_stamp 8.702802658s delta_ub -1.980724812s) ==== 2033+0+0 (crc 0 0 0) 0x3919980 con 0x396a000 2026-02-11T23:30:17.849+0000 7fac71bfd640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:17.849+0000 7fac71bfd640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:17.849+0000 7fac71bfd640 10 osd.0 21 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 14 to ping map epoch of 20 2026-02-11T23:30:17.849+0000 7fac71bfd640 10 osd.0 21 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 20 -> 21 (shared) 2026-02-11T23:30:17.849+0000 7fac71bfd640 10 osd.0 21 send_incremental_map epoch range: (21, 21] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:30:17.849+0000 7fac71bfd640 10 osd.0 21 add_map_inc_bl 21 618 bytes 2026-02-11T23:30:17.849+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(21..21 src has 1..21) -- 0x3941500 con 0x3752000 2026-02-11T23:30:17.849+0000 7fac723fe640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.981160998s,-1.980724812s]) 2026-02-11T23:30:17.849+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e21 up_from 5 ping_stamp 2026-02-11T23:30:17.849238+0000/8.702802658s send_stamp 10.683964729s delta_ub 1.981160998s) -- 0x3b1af00 con 0x396a000 2026-02-11T23:30:17.849+0000 7fac723fe640 20 osd.0 21 get_con_osd_cluster to osd.1 from_epoch 21 2026-02-11T23:30:17.849+0000 7fac723fe640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:17.849+0000 7fac723fe640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:17.849+0000 7fac723fe640 10 osd.0 21 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 21 is not newer than session's projected_epoch of 21 2026-02-11T23:30:17.978+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 1 ==== osd_ping(ping e19 up_from 17 ping_stamp 2026-02-11T23:30:17.978979+0000/3.503015995s send_stamp 3.503015995s) ==== 2025+0+0 (crc 0 0 0) 0x3b1a600 con 0x3f56800 2026-02-11T23:30:17.978+0000 7fac71bfd640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310427189s,]) 2026-02-11T23:30:17.978+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e21 up_from 5 ping_stamp 2026-02-11T23:30:17.978979+0000/3.503015995s send_stamp 10.813443184s delta_ub 7.310427189s) -- 0x3b1b080 con 0x3f56800 2026-02-11T23:30:17.978+0000 7fac71bfd640 20 osd.0 21 get_con_osd_cluster to osd.3 from_epoch 21 2026-02-11T23:30:17.978+0000 7fac71bfd640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:17.978+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] conn(0x3741000 0x3e37080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:17.978+0000 7fac71bfd640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:17.978+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 1 ==== osd_ping(ping e19 up_from 17 ping_stamp 2026-02-11T23:30:17.978979+0000/3.503015995s send_stamp 3.503015995s) ==== 2025+0+0 (crc 0 0 0) 0x3b1af00 con 0x3f56c00 2026-02-11T23:30:17.978+0000 7fac723fe640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310449123s,]) 2026-02-11T23:30:17.978+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e21 up_from 5 ping_stamp 2026-02-11T23:30:17.978979+0000/3.503015995s send_stamp 10.813466072s delta_ub 7.310449123s) -- 0x3919980 con 0x3f56c00 2026-02-11T23:30:17.978+0000 7fac723fe640 20 osd.0 21 get_con_osd_cluster to osd.3 from_epoch 21 2026-02-11T23:30:17.978+0000 7fac723fe640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:17.978+0000 7fac723fe640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:17.978+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] conn(0x3741000 0x3e37080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:17.978+0000 7fac71bfd640 10 monclient: get_auth_request con 0x3741000 auth_method 0 2026-02-11T23:30:17.978+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> conn(0x3741400 0x3e37600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:17.978+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> conn(0x3741400 0x3e37600 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:17.978+0000 7fac723fe640 10 osd.0 21 ms_handle_fast_authentication new session 0x3940c40 con 0x3741400 entity osd.3 addr 2026-02-11T23:30:17.978+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] conn(0x3741000 0x3e37080 unknown :-1 s=START_CONNECT pgs=0 gs=6 cs=0 l=0 c_cookie=911258765106907410 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2026-02-11T23:30:17.978+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] conn(0x3741400 0x3e37600 crc :-1 s=SESSION_ACCEPTING pgs=5 gs=0 cs=0 l=0 c_cookie=12449166567663491586 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_existing_connection connection race detected, replacing existing=0x3741000 socket by this connection's socket 2026-02-11T23:30:17.978+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] conn(0x3741400 0x3e37600 crc :-1 s=SESSION_ACCEPTING pgs=5 gs=0 cs=0 l=0 c_cookie=12449166567663491586 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:17.978+0000 7fac606e4640 2 osd.0 21 ms_handle_reset con 0x3741400 session 0x3940c40 2026-02-11T23:30:17.979+0000 7fac723fe640 10 osd.0 21 new session (incoming)0x3940e00 con=0x3741000 addr=v2:127.0.0.1:6828/1114810440 must have raced with connect 2026-02-11T23:30:17.979+0000 7fac723fe640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] conn(0x3741000 0x3e37080 unknown :-1 s=READY pgs=5 gs=7 cs=0 l=0 c_cookie=12449166567663491586 s_cookie=9935871053612673128 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=acc45308e7405a02 server_cookie=89e34e14acab6868 in_seq=0 out_seq=0 2026-02-11T23:30:18.010+0000 7fac60ee5640 20 osd.0 21 reports for 0 queries 2026-02-11T23:30:18.010+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x34c3c00 con 0x396a800 2026-02-11T23:30:18.010+0000 7fac60ee5640 15 osd.0 21 collect_pg_stats 2026-02-11T23:30:18.010+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836483 v 0) -- 0x3e48600 con 0x396a800 2026-02-11T23:30:18.104+0000 7fac68beb640 10 osd.0 21 tick 2026-02-11T23:30:18.104+0000 7fac68beb640 20 osd.0 21 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:18.154+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 1 ==== osd_ping(ping e21 up_from 21 ping_stamp 2026-02-11T23:30:18.155144+0000/1.102650523s send_stamp 1.102650523s) ==== 2025+0+0 (crc 0 0 0) 0x3b1a480 con 0x3f57400 2026-02-11T23:30:18.154+0000 7fac703fa640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886987686s,]) 2026-02-11T23:30:18.154+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e21 up_from 5 ping_stamp 2026-02-11T23:30:18.155144+0000/1.102650523s send_stamp 10.989637375s delta_ub 9.886987686s) -- 0x3b1ac00 con 0x3f57400 2026-02-11T23:30:18.154+0000 7fac703fa640 20 osd.0 21 get_con_osd_cluster to osd.4 from_epoch 21 2026-02-11T23:30:18.154+0000 7fac703fa640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:18.154+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 1 ==== osd_ping(ping e21 up_from 21 ping_stamp 2026-02-11T23:30:18.155144+0000/1.102650523s send_stamp 1.102650523s) ==== 2025+0+0 (crc 0 0 0) 0x3b1b080 con 0x3f57800 2026-02-11T23:30:18.154+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] conn(0x3e4a000 0x40ea000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-11T23:30:18.154+0000 7fac703fa640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:18.154+0000 7fac71bfd640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887025833s,]) 2026-02-11T23:30:18.154+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e21 up_from 5 ping_stamp 2026-02-11T23:30:18.155144+0000/1.102650523s send_stamp 10.989676476s delta_ub 9.887025833s) -- 0x3b1a600 con 0x3f57800 2026-02-11T23:30:18.154+0000 7fac71bfd640 20 osd.0 21 get_con_osd_cluster to osd.4 from_epoch 21 2026-02-11T23:30:18.154+0000 7fac71bfd640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:18.154+0000 7fac71bfd640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:18.154+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> conn(0x3741800 0x40eab00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:18.155+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> conn(0x3741800 0x40eab00 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:18.155+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] conn(0x3e4a000 0x40ea000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=8 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:18.155+0000 7fac71bfd640 10 monclient: get_auth_request con 0x3e4a000 auth_method 0 2026-02-11T23:30:18.155+0000 7fac703fa640 10 osd.0 21 ms_handle_fast_authentication new session 0x3941500 con 0x3741800 entity osd.4 addr 2026-02-11T23:30:18.155+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] conn(0x3741800 0x40eab00 crc :-1 s=SESSION_ACCEPTING pgs=1 gs=0 cs=0 l=0 c_cookie=14744292874227201248 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_existing_connection connection race detected, replacing existing=0x3e4a000 socket by this connection's socket 2026-02-11T23:30:18.155+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] conn(0x3741800 0x40eab00 crc :-1 s=SESSION_ACCEPTING pgs=1 gs=0 cs=0 l=0 c_cookie=14744292874227201248 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:18.155+0000 7fac606e4640 2 osd.0 21 ms_handle_reset con 0x3741800 session 0x3941500 2026-02-11T23:30:18.155+0000 7fac703fa640 10 osd.0 21 new session (incoming)0x3940380 con=0x3e4a000 addr=v2:127.0.0.1:6836/864727417 must have raced with connect 2026-02-11T23:30:18.155+0000 7fac703fa640 1 --2- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] >> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] conn(0x3e4a000 0x40ea000 unknown :-1 s=READY pgs=1 gs=9 cs=0 l=0 c_cookie=14744292874227201248 s_cookie=11956190471810977793 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=cc9e404020a2b4e0 server_cookie=a5ecebe333341001 in_seq=0 out_seq=0 2026-02-11T23:30:18.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:18.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:18.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:48.167972+0000) 2026-02-11T23:30:18.217+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 2 ==== osd_ping(ping e20 up_from 13 ping_stamp 2026-02-11T23:30:18.217489+0000/6.402884483s send_stamp 6.402884483s delta_ub -4.648889065s) ==== 2033+0+0 (crc 0 0 0) 0x3919980 con 0x3f56000 2026-02-11T23:30:18.217+0000 7fac723fe640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649082661s,-4.648889065s]) 2026-02-11T23:30:18.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 2 ==== osd_ping(ping e20 up_from 13 ping_stamp 2026-02-11T23:30:18.217489+0000/6.402884483s send_stamp 6.402884483s delta_ub -4.648889065s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ac00 con 0x3f56400 2026-02-11T23:30:18.217+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e21 up_from 5 ping_stamp 2026-02-11T23:30:18.217489+0000/6.402884483s send_stamp 11.051967621s delta_ub 4.649082661s) -- 0x3b1af00 con 0x3f56000 2026-02-11T23:30:18.217+0000 7fac723fe640 20 osd.0 21 get_con_osd_cluster to osd.2 from_epoch 21 2026-02-11T23:30:18.217+0000 7fac723fe640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:18.217+0000 7fac723fe640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:18.217+0000 7fac723fe640 10 osd.0 21 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 17 to ping map epoch of 20 2026-02-11T23:30:18.217+0000 7fac723fe640 10 osd.0 21 maybe_share_map: con v2:127.0.0.1:6820/2662569638 map epoch 20 -> 21 (shared) 2026-02-11T23:30:18.217+0000 7fac723fe640 10 osd.0 21 send_incremental_map epoch range: (21, 21] to v2:127.0.0.1:6820/2662569638 2026-02-11T23:30:18.217+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] -- osd_map(21..21 src has 1..21) -- 0x3940fc0 con 0x3f57c00 2026-02-11T23:30:18.217+0000 7fac703fa640 20 osd.0 21 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649120808s,-4.648889065s]) 2026-02-11T23:30:18.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e21 up_from 5 ping_stamp 2026-02-11T23:30:18.217489+0000/6.402884483s send_stamp 11.052005768s delta_ub 4.649120808s) -- 0x3b1a480 con 0x3f56400 2026-02-11T23:30:18.217+0000 7fac703fa640 20 osd.0 21 get_con_osd_cluster to osd.2 from_epoch 21 2026-02-11T23:30:18.217+0000 7fac703fa640 20 osd.0 21 get_nextmap_reserved map_reservations: {21=1} 2026-02-11T23:30:18.217+0000 7fac703fa640 20 osd.0 21 release_map epoch: 21 2026-02-11T23:30:18.217+0000 7fac703fa640 10 osd.0 21 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 21 is not newer than session's projected_epoch of 21 2026-02-11T23:30:18.262+0000 7fac683ea640 10 osd.0 21 tick_without_osd_lock 2026-02-11T23:30:18.262+0000 7fac683ea640 0 osd.0 21 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:18.262+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:18.262+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.180 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:18.262+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:18.263, recovery is active?:false restrictions:<....> 2026-02-11T23:30:18.262+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:18.262+0000 7fac683ea640 10 osd.0 21 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:18.262+0000 7fac683ea640 20 osd.0 21 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:18.262+0000 7fac683ea640 10 osd.0 21 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:18.262+0000 7fac683ea640 20 osd.0 21 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:18.777+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 3 ==== osd_map(22..24 src has 1..24) ==== 1139+0+0 (unknown 0 0 0) 0x3940540 con 0x3752000 2026-02-11T23:30:18.777+0000 7fac606e4640 20 osd.0 21 OSD::ms_dispatch: osd_map(22..24 src has 1..24) 2026-02-11T23:30:18.777+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 4 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 24) e24/24) ==== 158+0+0 (unknown 0 0 0) 0x3f3f680 con 0x3752000 2026-02-11T23:30:18.777+0000 7fac606e4640 20 osd.0 21 _dispatch 0x3940540 osd_map(22..24 src has 1..24) 2026-02-11T23:30:18.777+0000 7fac703fa640 15 osd.0 21 enqueue_peering_evt 1.0 epoch_sent: 24 epoch_requested: 24 MQuery 1.0 from 1 query_epoch 24 query: query(info 0'0 epoch_sent 24) 2026-02-11T23:30:18.777+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 24 epoch_requested: 24 MQuery 1.0 from 1 query_epoch 24 query: query(info 0'0 epoch_sent 24)) class_id 2 prio 255 cost 10 e24) 2026-02-11T23:30:18.777+0000 7fac606e4640 3 osd.0 21 handle_osd_map epochs [22,24], i have 21, src has [1,24] 2026-02-11T23:30:18.777+0000 7fac606e4640 10 osd.0 21 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:18.778+0000 7fac606e4640 10 osd.0 21 handle_osd_map got inc map for epoch 22 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 24 epoch_requested: 24 MQuery 1.0 from 1 query_epoch 24 query: query(info 0'0 epoch_sent 24)) class_id 2immediate prio 255 cost 10 e24) queued 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 24 epoch_requested: 24 MQuery 1.0 from 1 query_epoch 24 query: query(info 0'0 epoch_sent 24)) class_id 2immediate prio 255 cost 10 e24) pg 0 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 map 24 > 21 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 24, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 24 epoch_requested: 24 MQuery 1.0 from 1 query_epoch 24 query: query(info 0'0 epoch_sent 24)) class_id 2immediate prio 255 cost 10 e24) 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:18.778+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:18.778+0000 7fac606e4640 20 osd.0 21 got_full_map 22, nothing requested 2026-02-11T23:30:18.778+0000 7fac606e4640 10 osd.0 21 handle_osd_map got inc map for epoch 23 2026-02-11T23:30:18.778+0000 7fac606e4640 20 osd.0 21 got_full_map 23, nothing requested 2026-02-11T23:30:18.778+0000 7fac606e4640 10 osd.0 21 handle_osd_map got inc map for epoch 24 2026-02-11T23:30:18.778+0000 7fac606e4640 20 osd.0 21 got_full_map 24, nothing requested 2026-02-11T23:30:18.778+0000 7fac606e4640 10 osd.0 21 _track_pools_and_pg_num_changes recording new pool 1 pg_num 1 2026-02-11T23:30:18.778+0000 7fac606e4640 20 osd.0 21 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:18.778+0000 7fac606e4640 20 osd.0 21 handle_osd_map pg_num_history pg_num_history(e24 pg_nums {1={24=1}} deleted_pools ) 2026-02-11T23:30:18.778+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {22={},23={},24={}} 2026-02-11T23:30:18.778+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:18.778+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e24 maps [1~24] lci=[5,24] tlb=1) 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0 21 _committed_osd_maps 22..24 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0 21 advance to epoch 22 (<= last 24 <= newest_map 24) 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0 22 advance to epoch 23 (<= last 24 <= newest_map 24) 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0 23 advance to epoch 24 (<= last 24 <= newest_map 24) 2026-02-11T23:30:18.778+0000 7fac5e983640 0 osd.0 24 crush map has features 288514051259236352, adjusting msgr requires for clients 2026-02-11T23:30:18.778+0000 7fac5e983640 0 osd.0 24 crush map has features 288514051259236352 was 288514050185503233, adjusting msgr requires for mons 2026-02-11T23:30:18.778+0000 7fac5e983640 0 osd.0 24 crush map has features 3314933000852226048, adjusting msgr requires for osds 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0 24 consume_map version 24 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0 21 await_reserved_maps epoch:24 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0 21 await_reserved_maps done 0 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0 24 consume_map 24 -- publish done 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 1.0 has no pg and waiting_for_split 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0 24 prune_pg_created 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0:0.consume_map 24 (was 21) 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 24 <= 24, requeueing 2026-02-11T23:30:18.778+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {24=} 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0:1.consume_map 24 (was 21) 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0:2.consume_map 24 (was 21) 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0:3.consume_map 24 (was 21) 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0:4.consume_map 24 (was 21) 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0:5.consume_map 24 (was 21) 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0:6.consume_map 24 (was 21) 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0:7.consume_map 24 (was 21) 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 24 epoch_requested: 24 MQuery 1.0 from 1 query_epoch 24 query: query(info 0'0 epoch_sent 24)) class_id 2immediate prio 255 cost 10 e24) queued 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 24 epoch_requested: 24 MQuery 1.0 from 1 query_epoch 24 query: query(info 0'0 epoch_sent 24)) class_id 2immediate prio 255 cost 10 e24) pg 0 2026-02-11T23:30:18.778+0000 7fac51168640 10 osd.0 24 handle_pg_query_nopg 1.0 2026-02-11T23:30:18.778+0000 7fac51168640 10 osd.0 24 pg 1.0 dne 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 24 get_con_osd_cluster to osd.1 from_epoch 24 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 24 get_nextmap_reserved map_reservations: {24=1} 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 24 release_map epoch: 24 2026-02-11T23:30:18.778+0000 7fac51168640 10 osd.0 24 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 21 -> 24 (shared) 2026-02-11T23:30:18.778+0000 7fac51168640 10 osd.0 24 send_incremental_map epoch range: (22, 24] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:30:18.778+0000 7fac51168640 10 osd.0 24 add_map_inc_bl 22 244 bytes 2026-02-11T23:30:18.778+0000 7fac51168640 10 osd.0 24 add_map_inc_bl 23 244 bytes 2026-02-11T23:30:18.778+0000 7fac51168640 10 osd.0 24 add_map_inc_bl 24 591 bytes 2026-02-11T23:30:18.778+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(22..24 src has 1..24) -- 0x3941a40 con 0x3752000 2026-02-11T23:30:18.778+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:24 sent:24 1.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e24/24) -- 0x3380e00 con 0x3752000 2026-02-11T23:30:18.778+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:18.778+0000 7fac5e983640 10 osd.0 24 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:18.778+0000 7fac5e983640 7 osd.0 24 activate_map version 24 2026-02-11T23:30:18.778+0000 7fac55971640 20 osd.0 24 agent_entry empty queue 2026-02-11T23:30:18.877+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 5 ==== PGlog(1.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.731752396s int 16.000000000s) e25/24) ==== 1172+0+0 (unknown 0 0 0) 0x3903000 con 0x3752000 2026-02-11T23:30:18.877+0000 7fac703fa640 15 osd.0 24 enqueue_peering_evt 1.0 epoch_sent: 25 epoch_requested: 24 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.731752396s int 16.000000000s) +create_info 2026-02-11T23:30:18.877+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 24 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.731752396s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e25) 2026-02-11T23:30:18.877+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:18.877+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 24 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.731752396s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e25) queued 2026-02-11T23:30:18.877+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:18.877+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 24 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.731752396s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e25) pg 0 2026-02-11T23:30:18.877+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 map 25 > 24 2026-02-11T23:30:18.877+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 25, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 24 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.731752396s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e25) 2026-02-11T23:30:18.877+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:18.877+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.092+0000 7fac68beb640 10 osd.0 24 tick 2026-02-11T23:30:19.092+0000 7fac68beb640 20 osd.0 24 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:19.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:19.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:19.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:49.167997+0000) 2026-02-11T23:30:19.220+0000 7fac683ea640 10 osd.0 24 tick_without_osd_lock 2026-02-11T23:30:19.220+0000 7fac683ea640 0 osd.0 24 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:19.220+0000 7fac683ea640 20 osd.0 24 tick_without_osd_lock max_waiting_epoch 25, requesting new map 2026-02-11T23:30:19.220+0000 7fac683ea640 10 monclient: _renew_subs 2026-02-11T23:30:19.220+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:19.220+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=25}) -- 0x38f49c0 con 0x396bc00 2026-02-11T23:30:19.220+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:19.220+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:19.220+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:19.220+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.180 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:19.220+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:19.220, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:19.220+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:19.220+0000 7fac683ea640 10 osd.0 24 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:19.220+0000 7fac683ea640 20 osd.0 24 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:19.220+0000 7fac683ea640 10 osd.0 24 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:19.220+0000 7fac683ea640 20 osd.0 24 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:19.220+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 27 ==== osd_map(25..25 src has 1..25) ==== 296+0+0 (crc 0 0 0) 0x3941180 con 0x396bc00 2026-02-11T23:30:19.220+0000 7fac626e8640 20 osd.0 24 OSD::ms_dispatch: osd_map(25..25 src has 1..25) 2026-02-11T23:30:19.220+0000 7fac626e8640 20 osd.0 24 _dispatch 0x3941180 osd_map(25..25 src has 1..25) 2026-02-11T23:30:19.220+0000 7fac626e8640 3 osd.0 24 handle_osd_map epochs [25,25], i have 24, src has [1,25] 2026-02-11T23:30:19.220+0000 7fac626e8640 10 osd.0 24 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:19.220+0000 7fac626e8640 10 osd.0 24 handle_osd_map got inc map for epoch 25 2026-02-11T23:30:19.220+0000 7fac626e8640 20 osd.0 24 got_full_map 25, nothing requested 2026-02-11T23:30:19.220+0000 7fac626e8640 20 osd.0 24 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:19.220+0000 7fac626e8640 20 osd.0 24 handle_osd_map pg_num_history pg_num_history(e25 pg_nums {1={24=1}} deleted_pools ) 2026-02-11T23:30:19.220+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {25={}} 2026-02-11T23:30:19.220+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:19.220+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e25 maps [1~25] lci=[5,25] tlb=1) 2026-02-11T23:30:19.220+0000 7fac5e983640 10 osd.0 24 _committed_osd_maps 25..25 2026-02-11T23:30:19.220+0000 7fac5e983640 10 osd.0 24 advance to epoch 25 (<= last 25 <= newest_map 25) 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0 25 consume_map version 25 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0 24 await_reserved_maps epoch:25 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0 24 await_reserved_maps done 0 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0 25 consume_map 25 -- publish done 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 1.0 has no pg and waiting_for_split 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0 25 prune_pg_created 2026-02-11T23:30:19.220+0000 7fac5e983640 10 osd.0:0.consume_map 25 (was 24) 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 25 <= 25, requeueing 2026-02-11T23:30:19.220+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {25=} 2026-02-11T23:30:19.220+0000 7fac5e983640 10 osd.0:1.consume_map 25 (was 24) 2026-02-11T23:30:19.220+0000 7fac5e983640 10 osd.0:2.consume_map 25 (was 24) 2026-02-11T23:30:19.220+0000 7fac5e983640 10 osd.0:3.consume_map 25 (was 24) 2026-02-11T23:30:19.220+0000 7fac5e983640 10 osd.0:4.consume_map 25 (was 24) 2026-02-11T23:30:19.220+0000 7fac5e983640 10 osd.0:5.consume_map 25 (was 24) 2026-02-11T23:30:19.220+0000 7fac5e983640 10 osd.0:6.consume_map 25 (was 24) 2026-02-11T23:30:19.220+0000 7fac5e983640 10 osd.0:7.consume_map 25 (was 24) 2026-02-11T23:30:19.220+0000 7fac5e983640 10 osd.0 25 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:19.220+0000 7fac5e983640 7 osd.0 25 activate_map version 25 2026-02-11T23:30:19.220+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:19.220+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 24 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.731752396s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e25) queued 2026-02-11T23:30:19.220+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:19.220+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 24 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.731752396s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e25) pg 0 2026-02-11T23:30:19.220+0000 7fac55971640 20 osd.0 25 agent_entry empty queue 2026-02-11T23:30:19.220+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 no pg, should create on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 24 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.731752396s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e25) 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 25 _make_pg 1.0 2026-02-11T23:30:19.220+0000 7fac55170640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:30:19.220+0000 7fac55170640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:30:19.220+0000 7fac55170640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {"0000000000000001."}] update_bits prefix updated 2026-02-11T23:30:19.220+0000 7fac55170640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {"0000000000000001."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:30:19.220+0000 7fac55170640 5 osd.0 pg_epoch: 24 pg[1.0(unlocked)] enter Initial 2026-02-11T23:30:19.220+0000 7fac55170640 20 osd.0 pg_epoch: 24 pg[1.0(unlocked)] enter NotTrimming 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role 1 up [1,0,2] acting [1,0,2] history ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24 past_intervals ([0,0] all_participants={} intervals=[]) 2026-02-11T23:30:19.220+0000 7fac55170640 20 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:19.220+0000 7fac55170640 20 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [1,0,2]+[1,0,2] 2026-02-11T23:30:19.220+0000 7fac55170640 20 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x3533ab0] 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 12.055719376s + 0.000000000s) 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:19.220+0000 7fac55170640 15 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:19.220+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:19.220+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:19.220+0000 7fac55170640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:30:19.220+0000 7fac55170640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:30:19.220+0000 7fac55170640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:30:19.220+0000 7fac55170640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:30:19.220+0000 7fac55170640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:30:19.220+0000 7fac55170640 10 end of _write_log_and_missing 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:30:19.220+0000 7fac55170640 5 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] exit Initial 0.000163 0 0.000000 2026-02-11T23:30:19.220+0000 7fac55170640 5 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:19.220+0000 7fac55170640 20 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 24 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 24..24 2026-02-11T23:30:19.220+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:19.221+0000 7fac55170640 5 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] exit Reset 0.000032 1 0.000022 2026-02-11T23:30:19.221+0000 7fac55170640 5 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:30:19.221+0000 7fac55170640 5 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:30:19.221+0000 7fac55170640 1 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:30:19.221+0000 7fac55170640 5 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] exit Start 0.000032 0 0.000000 2026-02-11T23:30:19.221+0000 7fac55170640 5 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 24 while current is 24 2026-02-11T23:30:19.221+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 25 handle_pg_create_info new pg pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0:0._attach_pg 1.0 0x40f8000 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 25 identify_splits_and_merges 1.0 e24 to e25 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 25 identify_splits_and_merges 1.0 e24 to e25 pg_nums {24=1} 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0:0._prime_splits 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] handle_advance_map: 25 2026-02-11T23:30:19.221+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 24 -> 25 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 24 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [1,0,2]/[1,0,2] -- 1/1 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 25 advance_pg new pool opts old pool opts 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 25..25 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 24 while current is 25 2026-02-11T23:30:19.221+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 25 epoch_requested: 24 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 25.731752396s int 16.000000000s) +create_info 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] state: got info+log from osd.1 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:19.221+0000 7fac55170640 10 merge_log log((0'0,0'0], crt=0'0) from osd.1 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:19.221+0000 7fac55170640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:19.221+0000 7fac55170640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:19.221+0000 7fac55170640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 25.731752396s int 16.000000000s) 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.981160998s -> ruub 27.712913513s 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] PeeringState::proc_lease readable_until_ub now 27.712913513s 2026-02-11T23:30:19.221+0000 7fac55170640 5 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] exit Started/Stray 0.000142 3 0.000058 2026-02-11T23:30:19.221+0000 7fac55170640 5 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:19.221+0000 7fac55170640 5 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:19.221+0000 7fac55170640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:19.221+0000 7fac55170640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:30:19.221+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.221+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.221+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.221+0000 7fac55170640 15 osd.0 25 enqueue_peering_evt 1.0 epoch_sent: 25 epoch_requested: 25 ActivateCommitted from 25 processed at 25 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 ActivateCommitted from 25 processed at 25) class_id 2 prio 255 cost 10 e25) 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 ActivateCommitted from 25 processed at 25) class_id 2immediate prio 255 cost 10 e25) queued 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 ActivateCommitted from 25 processed at 25) class_id 2immediate prio 255 cost 10 e25) pg 0x40f8000 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 25 epoch_requested: 25 ActivateCommitted from 25 processed at 25 2026-02-11T23:30:19.221+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 unknown mbc={}] state: react 25 telling primary 2026-02-11T23:30:19.221+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 25 get_con_osd_cluster to osd.1 from_epoch 25 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 25 get_nextmap_reserved map_reservations: {25=1} 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 25 release_map epoch: 25 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 25 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 24 -> 25 (shared) 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 25 send_incremental_map epoch range: (25, 25] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:30:19.221+0000 7fac55170640 10 osd.0 25 add_map_inc_bl 25 252 bytes 2026-02-11T23:30:19.221+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(25..25 src has 1..25) -- 0x3941dc0 con 0x3752000 2026-02-11T23:30:19.221+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_info2(1.0 1.0 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/0 les/c/f=25/0/0 sis=24) e25/25) -- 0x3381500 con 0x3752000 2026-02-11T23:30:19.221+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.731+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 6 ==== pg_lease(1.0 pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s) e25/25) ==== 58+0+0 (unknown 0 0 0) 0x3b1a480 con 0x3752000 2026-02-11T23:30:19.731+0000 7fac703fa640 15 osd.0 25 enqueue_peering_evt 1.0 epoch_sent: 25 epoch_requested: 25 MLease epoch 25 from osd.1 pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s) 2026-02-11T23:30:19.732+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 MLease epoch 25 from osd.1 pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s)) class_id 2 prio 255 cost 10 e25) 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 MLease epoch 25 from osd.1 pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e25) queued 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 MLease epoch 25 from osd.1 pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e25) pg 0x40f8000 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.732+0000 7fac51168640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 25 epoch_requested: 25 MLease epoch 25 from osd.1 pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s) 2026-02-11T23:30:19.732+0000 7fac51168640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s) 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -1.980724812s -> ru 27.712476730s 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 27.712476730s 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.981160998s -> ruub 28.567081451s 2026-02-11T23:30:19.732+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 7 ==== pg_info2(1.0 1.0 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/0 les/c/f=25/0/0 sis=24) e25/25) ==== 1126+0+0 (unknown 0 0 0) 0x3380e00 con 0x3752000 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 28.567081451s 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 25 get_con_osd_cluster to osd.1 from_epoch 25 2026-02-11T23:30:19.732+0000 7fac703fa640 15 osd.0 25 enqueue_peering_evt 1.0 epoch_sent: 25 epoch_requested: 25 MInfoRec from 1 info: 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/0 les/c/f=25/0/0 sis=24) pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s) 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 25 get_nextmap_reserved map_reservations: {25=1} 2026-02-11T23:30:19.732+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 MInfoRec from 1 info: 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/0 les/c/f=25/0/0 sis=24) pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s)) class_id 2 prio 255 cost 10 e25) 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 25 release_map epoch: 25 2026-02-11T23:30:19.732+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_lease_ack(1.0 pg_lease_ack(ruub 26.585920334s) e25/25) -- 0x2580000 con 0x3752000 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 MInfoRec from 1 info: 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/0 les/c/f=25/0/0 sis=24) pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e25) queued 2026-02-11T23:30:19.732+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 MInfoRec from 1 info: 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/0 les/c/f=25/0/0 sis=24) pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e25) pg 0x40f8000 2026-02-11T23:30:19.732+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=0/0 les/c/f=0/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 25 epoch_requested: 25 MInfoRec from 1 info: 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/0 les/c/f=25/0/0 sis=24) pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s) 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/0 les/c/f=25/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=24/24 lis/c=24/0 les/c/f=25/0/0 sis=24 2026-02-11T23:30:19.732+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.732+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 8 ==== pg_info2(1.0 1.0 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) e25/25) ==== 1126+0+0 (unknown 0 0 0) 0x3380e00 con 0x3752000 2026-02-11T23:30:19.732+0000 7fac703fa640 15 osd.0 25 enqueue_peering_evt 1.0 epoch_sent: 25 epoch_requested: 25 MInfoRec from 1 info: 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s) 2026-02-11T23:30:19.732+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 MInfoRec from 1 info: 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s)) class_id 2 prio 255 cost 10 e25) 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 MInfoRec from 1 info: 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e25) queued 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 25 epoch_requested: 25 MInfoRec from 1 info: 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e25) pg 0x40f8000 2026-02-11T23:30:19.732+0000 7fac55170640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/0 les/c/f=25/0/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 25 epoch_requested: 25 MInfoRec from 1 info: 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) pg_lease(ru 25.731752396s ub 26.585920334s int 16.000000000s) 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:19.732+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.732+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:19.732+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.073+0000 7fac68beb640 10 osd.0 25 tick 2026-02-11T23:30:20.073+0000 7fac68beb640 20 osd.0 25 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:20.086+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 9 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 27) e27/27) ==== 158+0+0 (unknown 0 0 0) 0x3f3f680 con 0x3752000 2026-02-11T23:30:20.086+0000 7fac703fa640 15 osd.0 25 enqueue_peering_evt 1.0 epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27) 2026-02-11T23:30:20.086+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2 prio 255 cost 10 e27) 2026-02-11T23:30:20.086+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.086+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2immediate prio 255 cost 10 e27) queued 2026-02-11T23:30:20.086+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 10 ==== osd_map(25..27 src has 1..27) ==== 1460+0+0 (unknown 0 0 0) 0x3941dc0 con 0x3752000 2026-02-11T23:30:20.086+0000 7fac606e4640 20 osd.0 25 OSD::ms_dispatch: osd_map(25..27 src has 1..27) 2026-02-11T23:30:20.086+0000 7fac606e4640 20 osd.0 25 _dispatch 0x3941dc0 osd_map(25..27 src has 1..27) 2026-02-11T23:30:20.086+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.086+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2immediate prio 255 cost 10 e27) pg 0x40f8000 2026-02-11T23:30:20.086+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 27, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2immediate prio 255 cost 10 e27) 2026-02-11T23:30:20.086+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.086+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.087+0000 7fac606e4640 3 osd.0 25 handle_osd_map epochs [25,27], i have 25, src has [1,27] 2026-02-11T23:30:20.087+0000 7fac606e4640 10 osd.0 25 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:20.087+0000 7fac606e4640 10 osd.0 25 handle_osd_map got inc map for epoch 26 2026-02-11T23:30:20.087+0000 7fac606e4640 20 osd.0 25 got_full_map 26, nothing requested 2026-02-11T23:30:20.087+0000 7fac606e4640 10 osd.0 25 handle_osd_map got inc map for epoch 27 2026-02-11T23:30:20.087+0000 7fac606e4640 20 osd.0 25 got_full_map 27, nothing requested 2026-02-11T23:30:20.087+0000 7fac606e4640 20 osd.0 25 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:20.087+0000 7fac606e4640 20 osd.0 25 handle_osd_map pg_num_history pg_num_history(e27 pg_nums {1={24=1}} deleted_pools ) 2026-02-11T23:30:20.087+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {26={},27={}} 2026-02-11T23:30:20.087+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:20.087+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e27 maps [1~27] lci=[5,27] tlb=1) 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0 25 _committed_osd_maps 26..27 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0 25 advance to epoch 26 (<= last 27 <= newest_map 27) 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0 26 advance to epoch 27 (<= last 27 <= newest_map 27) 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0 27 consume_map version 27 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0 25 await_reserved_maps epoch:27 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0 25 await_reserved_maps done 0 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0 27 consume_map 27 -- publish done 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0 27 identify_splits_and_merges 1.0 e25 to e27 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0 27 identify_splits_and_merges 1.0 e25 to e27 pg_nums {24=1} 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0 27 prune_pg_created 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0:0.consume_map 27 (was 25) 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 27 <= 27, requeueing 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {27=} 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0:1.consume_map 27 (was 25) 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0:2.consume_map 27 (was 25) 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0:3.consume_map 27 (was 25) 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0:4.consume_map 27 (was 25) 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0:5.consume_map 27 (was 25) 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0:6.consume_map 27 (was 25) 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0:7.consume_map 27 (was 25) 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2immediate prio 255 cost 10 e27) queued 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27)) class_id 2immediate prio 255 cost 10 e27) pg 0x40f8000 2026-02-11T23:30:20.087+0000 7fac5e983640 10 osd.0 27 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:20.087+0000 7fac5e983640 15 osd.0 27 enqueue_peering_evt 1.0 epoch_sent: 27 epoch_requested: 27 NullEvt 2026-02-11T23:30:20.087+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 NullEvt) class_id 2 prio 255 cost 10 e27) 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] handle_advance_map: 26 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 25 -> 26 2026-02-11T23:30:20.087+0000 7fac5e983640 7 osd.0 27 activate_map version 27 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 25 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0,2]/[1,0,2] -- 1/1 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 26 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:20.087+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.087+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 NullEvt) class_id 2immediate prio 255 cost 10 e27) queued 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 26 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 26 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 26 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:20.087+0000 7fac55971640 20 osd.0 27 agent_entry empty queue 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 26 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 27 advance_pg new pool opts old pool opts 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 26 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] handle_advance_map: 27 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 26 -> 27 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 26 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [1,0] newacting [1,0] 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:20.087+0000 7fac51168640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.866321 6 0.000111 2026-02-11T23:30:20.087+0000 7fac51168640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] exit Started/ReplicaActive 0.866327 0 0.000000 2026-02-11T23:30:20.087+0000 7fac51168640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] exit Started 0.866514 0 0.000000 2026-02-11T23:30:20.087+0000 7fac51168640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=24 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 27 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=27 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [1,0] newacting [1,0] 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=27 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0,2] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 27 2026-02-11T23:30:20.087+0000 7fac51168640 15 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(24-26 up [1, 0, 2](1) acting [1, 0, 2](1)) up_thru 24 up_from 9 last_epoch_clean 25 interval(24-26 up [1, 0, 2](1) acting [1, 0, 2](1) maybe_went_rw) : primary up 9-24 includes interval 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([24,26] all_participants={0, 1, 2} intervals=[([24,26] acting={0, 1, 2})]) 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [1,0]+[1,0] 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [0x3533ab0] 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 28.567081451s (mnow 12.922386169s + 15.644694328s) 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] cancel_recovery 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] clear_recovery_state 2026-02-11T23:30:20.087+0000 7fac51168640 15 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:20.087+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:20.087+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:20.087+0000 7fac51168640 1 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] PeeringState::start_peering_interval up [1,0,2] -> [1,0], acting [1,0,2] -> [1,0], acting_primary 1 -> 1, up_primary 1 -> 1, role 1 -> 1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] clear_primary_state 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] release_backoffs [1:00000000::::0,MAX) 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:20.087+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:20.087+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] agent_stop 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] on_change 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] on_change_cleanup 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] on_change 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] exit NotTrimming 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] enter NotTrimming 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] hit_set_clear 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] agent_stop 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 27 advance_pg new pool opts old pool opts 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] handle_activate_map: epoch range: 26..27 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644694328s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:20.087+0000 7fac51168640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644647598s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] exit Reset 0.000083 1 0.000086 2026-02-11T23:30:20.087+0000 7fac51168640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644647598s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] enter Started 2026-02-11T23:30:20.087+0000 7fac51168640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644647598s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] enter Start 2026-02-11T23:30:20.087+0000 7fac51168640 1 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644647598s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:20.087+0000 7fac51168640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644647598s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:20.087+0000 7fac51168640 5 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644647598s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] enter Started/Stray 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644647598s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 25 while current is 27 2026-02-11T23:30:20.087+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644647598s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644647598s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] do_peering_event: epoch_sent: 27 epoch_requested: 27 MQuery 1.0 from 1 query_epoch 27 query: query(info 0'0 epoch_sent 27) 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:20.087+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 27 get_con_osd_cluster to osd.1 from_epoch 27 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 27 get_nextmap_reserved map_reservations: {27=1} 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 27 release_map epoch: 27 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 27 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 25 -> 27 (shared) 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 27 send_incremental_map epoch range: (26, 27] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 27 add_map_inc_bl 26 574 bytes 2026-02-11T23:30:20.087+0000 7fac51168640 10 osd.0 27 add_map_inc_bl 27 574 bytes 2026-02-11T23:30:20.087+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(26..27 src has 1..27) -- 0x3940700 con 0x3752000 2026-02-11T23:30:20.087+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:27 sent:27 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644647598s) ([24,26] all_participants={0, 1, 2} intervals=[([24,26] acting={0, 1, 2})])) e27/27) -- 0x3382300 con 0x3752000 2026-02-11T23:30:20.087+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:27 sent:27 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) ([24,26] all_participants={0, 1, 2} intervals=[([24,26] acting={0, 1, 2})])) e27/27) -- 0x3382a00 con 0x3752000 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.087+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.087+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 27 epoch_requested: 27 NullEvt) class_id 2immediate prio 255 cost 10 e27) pg 0x40f8000 2026-02-11T23:30:20.087+0000 7fac55170640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] do_peering_event: epoch_sent: 27 epoch_requested: 27 NullEvt 2026-02-11T23:30:20.087+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:20.087+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.087+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.087+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:20.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:20.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:50.168032+0000) 2026-02-11T23:30:20.167+0000 7fac616e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:20.186+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 11 ==== PGlog(1.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.040706635s int 16.000000000s) e28/27) ==== 1164+0+0 (unknown 0 0 0) 0x3902800 con 0x3752000 2026-02-11T23:30:20.186+0000 7fac703fa640 15 osd.0 27 enqueue_peering_evt 1.0 epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.040706635s int 16.000000000s) +create_info 2026-02-11T23:30:20.186+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.040706635s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e28) 2026-02-11T23:30:20.186+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.186+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.040706635s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e28) queued 2026-02-11T23:30:20.186+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.186+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.040706635s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e28) pg 0x40f8000 2026-02-11T23:30:20.186+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 28, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.040706635s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e28) 2026-02-11T23:30:20.186+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.186+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.216+0000 7fac683ea640 10 osd.0 27 tick_without_osd_lock 2026-02-11T23:30:20.216+0000 7fac683ea640 0 osd.0 27 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:20.216+0000 7fac683ea640 20 osd.0 27 tick_without_osd_lock max_waiting_epoch 28, requesting new map 2026-02-11T23:30:20.216+0000 7fac683ea640 10 monclient: _renew_subs 2026-02-11T23:30:20.216+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:20.216+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=28}) -- 0x38f5d40 con 0x396bc00 2026-02-11T23:30:20.216+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:20.216+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.166 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:20.216+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:20.216, recovery is active?:false restrictions:<....> 2026-02-11T23:30:20.216+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:20.216+0000 7fac683ea640 10 osd.0 27 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:20.216+0000 7fac683ea640 20 osd.0 27 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:20.216+0000 7fac683ea640 10 osd.0 27 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:20.216+0000 7fac683ea640 20 osd.0 27 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:20.216+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 28 ==== osd_map(28..28 src has 1..28) ==== 626+0+0 (crc 0 0 0) 0x38e3880 con 0x396bc00 2026-02-11T23:30:20.216+0000 7fac626e8640 20 osd.0 27 OSD::ms_dispatch: osd_map(28..28 src has 1..28) 2026-02-11T23:30:20.216+0000 7fac626e8640 20 osd.0 27 _dispatch 0x38e3880 osd_map(28..28 src has 1..28) 2026-02-11T23:30:20.216+0000 7fac626e8640 3 osd.0 27 handle_osd_map epochs [28,28], i have 27, src has [1,28] 2026-02-11T23:30:20.216+0000 7fac626e8640 10 osd.0 27 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:20.216+0000 7fac626e8640 10 osd.0 27 handle_osd_map got inc map for epoch 28 2026-02-11T23:30:20.216+0000 7fac626e8640 20 osd.0 27 got_full_map 28, nothing requested 2026-02-11T23:30:20.216+0000 7fac626e8640 20 osd.0 27 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:20.216+0000 7fac626e8640 20 osd.0 27 handle_osd_map pg_num_history pg_num_history(e28 pg_nums {1={24=1}} deleted_pools ) 2026-02-11T23:30:20.216+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {28={}} 2026-02-11T23:30:20.216+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:20.216+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e28 maps [1~28] lci=[5,28] tlb=1) 2026-02-11T23:30:20.216+0000 7fac5e983640 10 osd.0 27 _committed_osd_maps 28..28 2026-02-11T23:30:20.216+0000 7fac5e983640 10 osd.0 27 advance to epoch 28 (<= last 28 <= newest_map 28) 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0 28 consume_map version 28 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0 27 await_reserved_maps epoch:28 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0 27 await_reserved_maps done 0 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0 28 consume_map 28 -- publish done 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0 28 identify_splits_and_merges 1.0 e27 to e28 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0 28 identify_splits_and_merges 1.0 e27 to e28 pg_nums {24=1} 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0 28 prune_pg_created 2026-02-11T23:30:20.216+0000 7fac5e983640 10 osd.0:0.consume_map 28 (was 27) 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 28 <= 28, requeueing 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {28=} 2026-02-11T23:30:20.216+0000 7fac5e983640 10 osd.0:1.consume_map 28 (was 27) 2026-02-11T23:30:20.216+0000 7fac5e983640 10 osd.0:2.consume_map 28 (was 27) 2026-02-11T23:30:20.216+0000 7fac5e983640 10 osd.0:3.consume_map 28 (was 27) 2026-02-11T23:30:20.216+0000 7fac5e983640 10 osd.0:4.consume_map 28 (was 27) 2026-02-11T23:30:20.216+0000 7fac5e983640 10 osd.0:5.consume_map 28 (was 27) 2026-02-11T23:30:20.216+0000 7fac5e983640 10 osd.0:6.consume_map 28 (was 27) 2026-02-11T23:30:20.216+0000 7fac5e983640 10 osd.0:7.consume_map 28 (was 27) 2026-02-11T23:30:20.216+0000 7fac5e983640 10 osd.0 28 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.216+0000 7fac5e983640 15 osd.0 28 enqueue_peering_evt 1.0 epoch_sent: 28 epoch_requested: 28 NullEvt 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.040706635s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e28) queued 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.040706635s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e28) pg 0x40f8000 2026-02-11T23:30:20.216+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 NullEvt) class_id 2 prio 255 cost 10 e28) 2026-02-11T23:30:20.216+0000 7fac5e983640 7 osd.0 28 activate_map version 28 2026-02-11T23:30:20.216+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.216+0000 7fac55971640 20 osd.0 28 agent_entry empty queue 2026-02-11T23:30:20.216+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 NullEvt) class_id 2immediate prio 255 cost 10 e28) queued 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] handle_advance_map: 28 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 27 -> 28 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 27 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] state: Started advmap 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] hit_set_clear 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] agent_stop 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 28 advance_pg new pool opts old pool opts 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] handle_activate_map: epoch range: 28..28 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.644625664s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 27 while current is 28 2026-02-11T23:30:20.216+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] do_peering_event: epoch_sent: 28 epoch_requested: 27 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.040706635s int 16.000000000s) +create_info 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] state: got info+log from osd.1 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:20.216+0000 7fac51168640 10 merge_log log((0'0,0'0], crt=0'0) from osd.1 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:20.216+0000 7fac51168640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:20.216+0000 7fac51168640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:20.216+0000 7fac51168640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 27.040706635s int 16.000000000s) 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.981160998s -> ruub 29.021867752s 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] PeeringState::proc_lease readable_until_ub now 29.021867752s 2026-02-11T23:30:20.216+0000 7fac51168640 5 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] exit Started/Stray 0.129328 5 0.000025 2026-02-11T23:30:20.216+0000 7fac51168640 5 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:20.216+0000 7fac51168640 5 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown NOTIFY pruub 28.567081451s@ mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:20.216+0000 7fac51168640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:20.216+0000 7fac51168640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] state: Activate Finished 2026-02-11T23:30:20.216+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 28 get_con_osd_cluster to osd.1 from_epoch 28 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 28 get_nextmap_reserved map_reservations: {28=1} 2026-02-11T23:30:20.216+0000 7fac51168640 20 osd.0 28 release_map epoch: 28 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 28 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 27 -> 28 (shared) 2026-02-11T23:30:20.216+0000 7fac51168640 10 osd.0 28 send_incremental_map epoch range: (28, 28] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:30:20.217+0000 7fac51168640 10 osd.0 28 add_map_inc_bl 28 582 bytes 2026-02-11T23:30:20.217+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(28..28 src has 1..28) -- 0x38e21c0 con 0x3752000 2026-02-11T23:30:20.217+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:28 sent:28 1.0( empty local-lis/les=24/25 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) ([24,26] all_participants={0, 1, 2} intervals=[([24,26] acting={0, 1, 2})])) e28/28) -- 0x3383100 con 0x3752000 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 NullEvt) class_id 2immediate prio 255 cost 10 e28) pg 0x40f8000 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac55170640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] do_peering_event: epoch_sent: 28 epoch_requested: 28 NullEvt 2026-02-11T23:30:20.217+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac55170640 15 osd.0 28 enqueue_peering_evt 1.0 epoch_sent: 28 epoch_requested: 28 ActivateCommitted from 28 processed at 28 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 ActivateCommitted from 28 processed at 28) class_id 2 prio 255 cost 10 e28) 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 ActivateCommitted from 28 processed at 28) class_id 2immediate prio 255 cost 10 e28) queued 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 ActivateCommitted from 28 processed at 28) class_id 2immediate prio 255 cost 10 e28) pg 0x40f8000 2026-02-11T23:30:20.217+0000 7fac55170640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] do_peering_event: epoch_sent: 28 epoch_requested: 28 ActivateCommitted from 28 processed at 28 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac55170640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 unknown pruub 28.567081451s@ mbc={}] state: react 28 telling primary 2026-02-11T23:30:20.217+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 28 get_con_osd_cluster to osd.1 from_epoch 28 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 28 get_nextmap_reserved map_reservations: {28=1} 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 28 release_map epoch: 28 2026-02-11T23:30:20.217+0000 7fac55170640 10 osd.0 28 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 28 is not newer than session's projected_epoch of 28 2026-02-11T23:30:20.217+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_info2(1.0 1.0 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/24 les/c/f=28/25/0 sis=27 pruub=15.515338898s) e28/28) -- 0x3381500 con 0x3752000 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 12 ==== pg_lease(1.0 pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s) e28/28) ==== 58+0+0 (unknown 0 0 0) 0x3b1a480 con 0x3752000 2026-02-11T23:30:20.217+0000 7fac703fa640 15 osd.0 28 enqueue_peering_evt 1.0 epoch_sent: 28 epoch_requested: 28 MLease epoch 28 from osd.1 pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s) 2026-02-11T23:30:20.217+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 MLease epoch 28 from osd.1 pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s)) class_id 2 prio 255 cost 10 e28) 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 MLease epoch 28 from osd.1 pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e28) queued 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 MLease epoch 28 from osd.1 pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e28) pg 0x40f8000 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] do_peering_event: epoch_sent: 28 epoch_requested: 28 MLease epoch 28 from osd.1 pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s) 2026-02-11T23:30:20.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 13 ==== pg_info2(1.0 1.0 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/24 les/c/f=28/25/0 sis=27) e28/28) ==== 1118+0+0 (unknown 0 0 0) 0x3381500 con 0x3752000 2026-02-11T23:30:20.217+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] PeeringState::proc_lease pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s) 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] PeeringState::proc_lease peer_clock_delta_ub -1.980724812s -> ru 29.021430969s 2026-02-11T23:30:20.217+0000 7fac703fa640 15 osd.0 28 enqueue_peering_evt 1.0 epoch_sent: 28 epoch_requested: 28 MInfoRec from 1 info: 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/24 les/c/f=28/25/0 sis=27) pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s) 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] PeeringState::proc_lease readable_until now 29.021430969s 2026-02-11T23:30:20.217+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 MInfoRec from 1 info: 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/24 les/c/f=28/25/0 sis=27) pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s)) class_id 2 prio 255 cost 10 e28) 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.981160998s -> ruub 29.052526474s 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] PeeringState::proc_lease readable_until_ub now 29.052526474s 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 28 get_con_osd_cluster to osd.1 from_epoch 28 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 28 get_nextmap_reserved map_reservations: {28=1} 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 28 release_map epoch: 28 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 MInfoRec from 1 info: 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/24 les/c/f=28/25/0 sis=27) pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e28) queued 2026-02-11T23:30:20.217+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_lease_ack(1.0 pg_lease_ack(ruub 27.071365356s) e28/28) -- 0x25802c0 con 0x3752000 2026-02-11T23:30:20.217+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 MInfoRec from 1 info: 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/24 les/c/f=28/25/0 sis=27) pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e28) pg 0x40f8000 2026-02-11T23:30:20.217+0000 7fac55170640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=27 pruub=15.515338898s) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] do_peering_event: epoch_sent: 28 epoch_requested: 28 MInfoRec from 1 info: 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/24 les/c/f=28/25/0 sis=27) pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s) 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/24 les/c/f=28/25/0 sis=27) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active pruub 28.567081451s@ mbc={}] PeeringState::update_history advanced history from ec=24/24 lis/c=27/24 les/c/f=28/25/0 sis=27 2026-02-11T23:30:20.217+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 14 ==== pg_info2(1.0 1.0 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) e28/28) ==== 1118+0+0 (unknown 0 0 0) 0x3381500 con 0x3752000 2026-02-11T23:30:20.217+0000 7fac703fa640 15 osd.0 28 enqueue_peering_evt 1.0 epoch_sent: 28 epoch_requested: 28 MInfoRec from 1 info: 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s) 2026-02-11T23:30:20.217+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 MInfoRec from 1 info: 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s)) class_id 2 prio 255 cost 10 e28) 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 MInfoRec from 1 info: 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e28) queued 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 28 epoch_requested: 28 MInfoRec from 1 info: 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e28) pg 0x40f8000 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac51168640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/24 les/c/f=28/25/0 sis=27) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 28 epoch_requested: 28 MInfoRec from 1 info: 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) pg_lease(ru 27.040706635s ub 27.071365356s int 16.000000000s) 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 pi=[24,27)/1 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:20.217+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.489+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 29 ==== osd_map(29..30 src has 1..30) ==== 1200+0+0 (crc 0 0 0) 0x3940540 con 0x396bc00 2026-02-11T23:30:20.489+0000 7fac626e8640 20 osd.0 28 OSD::ms_dispatch: osd_map(29..30 src has 1..30) 2026-02-11T23:30:20.489+0000 7fac626e8640 20 osd.0 28 _dispatch 0x3940540 osd_map(29..30 src has 1..30) 2026-02-11T23:30:20.489+0000 7fac626e8640 3 osd.0 28 handle_osd_map epochs [29,30], i have 28, src has [1,30] 2026-02-11T23:30:20.489+0000 7fac626e8640 10 osd.0 28 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:20.489+0000 7fac626e8640 10 osd.0 28 handle_osd_map got inc map for epoch 29 2026-02-11T23:30:20.489+0000 7fac626e8640 20 osd.0 28 got_full_map 29, nothing requested 2026-02-11T23:30:20.489+0000 7fac626e8640 10 osd.0 28 handle_osd_map got inc map for epoch 30 2026-02-11T23:30:20.489+0000 7fac626e8640 20 osd.0 28 got_full_map 30, nothing requested 2026-02-11T23:30:20.489+0000 7fac626e8640 20 osd.0 28 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:20.489+0000 7fac626e8640 20 osd.0 28 handle_osd_map pg_num_history pg_num_history(e30 pg_nums {1={24=1}} deleted_pools ) 2026-02-11T23:30:20.489+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {29={},30={}} 2026-02-11T23:30:20.489+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:20.489+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e30 maps [1~30] lci=[5,30] tlb=1) 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0 28 _committed_osd_maps 29..30 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0 28 advance to epoch 29 (<= last 30 <= newest_map 30) 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0 29 advance to epoch 30 (<= last 30 <= newest_map 30) 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0 30 consume_map version 30 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0 28 await_reserved_maps epoch:30 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0 28 await_reserved_maps done 0 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0 30 consume_map 30 -- publish done 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0 30 identify_splits_and_merges 1.0 e28 to e30 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0 30 identify_splits_and_merges 1.0 e28 to e30 pg_nums {24=1} 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0 30 prune_pg_created 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0:0.consume_map 30 (was 28) 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0:1.consume_map 30 (was 28) 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0:2.consume_map 30 (was 28) 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0:3.consume_map 30 (was 28) 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0:4.consume_map 30 (was 28) 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0:5.consume_map 30 (was 28) 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0:6.consume_map 30 (was 28) 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0:7.consume_map 30 (was 28) 2026-02-11T23:30:20.490+0000 7fac5e983640 10 osd.0 30 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:20.490+0000 7fac5e983640 15 osd.0 30 enqueue_peering_evt 1.0 epoch_sent: 30 epoch_requested: 30 NullEvt 2026-02-11T23:30:20.490+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 NullEvt) class_id 2 prio 255 cost 10 e30) 2026-02-11T23:30:20.490+0000 7fac5e983640 7 osd.0 30 activate_map version 30 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.490+0000 7fac55971640 20 osd.0 30 agent_entry empty queue 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 NullEvt) class_id 2immediate prio 255 cost 10 e30) queued 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 30 epoch_requested: 30 NullEvt) class_id 2immediate prio 255 cost 10 e30) pg 0x40f8000 2026-02-11T23:30:20.490+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 29 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 28 -> 29 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 28 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 30 advance_pg new pool opts old pool opts 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 30 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 29 -> 30 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 29 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 30 advance_pg new pool opts old pool opts 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 29..30 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 30 2026-02-11T23:30:20.490+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:20.490+0000 7fac55170640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 30 epoch_requested: 30 NullEvt 2026-02-11T23:30:20.490+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:20.490+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.879+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 2 ==== osd_ping(ping e26 up_from 17 ping_stamp 2026-02-11T23:30:20.879141+0000/6.403178215s send_stamp 6.403178215s delta_ub -7.310327530s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a600 con 0x3f56800 2026-02-11T23:30:20.879+0000 7fac71bfd640 20 osd.0 30 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310462952s,-7.310327530s]) 2026-02-11T23:30:20.879+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e30 up_from 5 ping_stamp 2026-02-11T23:30:20.879141+0000/6.403178215s send_stamp 13.713640213s delta_ub 7.310462952s) -- 0x3b1b080 con 0x3f56800 2026-02-11T23:30:20.879+0000 7fac71bfd640 20 osd.0 30 get_con_osd_cluster to osd.3 from_epoch 30 2026-02-11T23:30:20.879+0000 7fac71bfd640 20 osd.0 30 get_nextmap_reserved map_reservations: {30=1} 2026-02-11T23:30:20.879+0000 7fac71bfd640 20 osd.0 30 release_map epoch: 30 2026-02-11T23:30:20.879+0000 7fac71bfd640 10 osd.0 30 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 0 to ping map epoch of 26 2026-02-11T23:30:20.879+0000 7fac71bfd640 10 osd.0 30 maybe_share_map: con v2:127.0.0.1:6828/1114810440 map epoch 26 -> 30 (shared) 2026-02-11T23:30:20.879+0000 7fac71bfd640 10 osd.0 30 send_incremental_map epoch range: (27, 30] to v2:127.0.0.1:6828/1114810440 2026-02-11T23:30:20.879+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 2 ==== osd_ping(ping e26 up_from 17 ping_stamp 2026-02-11T23:30:20.879141+0000/6.403178215s send_stamp 6.403178215s delta_ub -7.310327530s) ==== 2033+0+0 (crc 0 0 0) 0x3919980 con 0x3f56c00 2026-02-11T23:30:20.879+0000 7fac71bfd640 10 osd.0 30 add_map_inc_bl 29 574 bytes 2026-02-11T23:30:20.879+0000 7fac71bfd640 10 osd.0 30 add_map_inc_bl 30 574 bytes 2026-02-11T23:30:20.879+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- osd_map(27..30 src has 1..30) -- 0x3940fc0 con 0x3741000 2026-02-11T23:30:20.879+0000 7fac723fe640 20 osd.0 30 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310860634s,-7.310327530s]) 2026-02-11T23:30:20.879+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e30 up_from 5 ping_stamp 2026-02-11T23:30:20.879141+0000/6.403178215s send_stamp 13.714038849s delta_ub 7.310860634s) -- 0x3b1a900 con 0x3f56c00 2026-02-11T23:30:20.879+0000 7fac723fe640 20 osd.0 30 get_con_osd_cluster to osd.3 from_epoch 30 2026-02-11T23:30:20.879+0000 7fac723fe640 20 osd.0 30 get_nextmap_reserved map_reservations: {30=1} 2026-02-11T23:30:20.879+0000 7fac723fe640 20 osd.0 30 release_map epoch: 30 2026-02-11T23:30:20.879+0000 7fac723fe640 10 osd.0 30 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 30 is not newer than session's projected_epoch of 30 2026-02-11T23:30:20.880+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 1 ==== osd_map(31..31 src has 1..31) ==== 635+0+0 (unknown 0 0 0) 0x3940fc0 con 0x3741000 2026-02-11T23:30:20.880+0000 7fac606e4640 20 osd.0 30 OSD::ms_dispatch: osd_map(31..31 src has 1..31) 2026-02-11T23:30:20.880+0000 7fac606e4640 20 osd.0 30 _dispatch 0x3940fc0 osd_map(31..31 src has 1..31) 2026-02-11T23:30:20.880+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 2 ==== pg_query2(2.0 2.0 query(info 0'0 epoch_sent 31) e31/31) ==== 158+0+0 (unknown 0 0 0) 0x3f3f0e0 con 0x3741000 2026-02-11T23:30:20.880+0000 7fac723fe640 15 osd.0 30 enqueue_peering_evt 2.0 epoch_sent: 31 epoch_requested: 31 MQuery 2.0 from 3 query_epoch 31 query: query(info 0'0 epoch_sent 31) 2026-02-11T23:30:20.880+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 31 epoch_requested: 31 MQuery 2.0 from 3 query_epoch 31 query: query(info 0'0 epoch_sent 31)) class_id 2 prio 255 cost 10 e31) 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.880+0000 7fac606e4640 3 osd.0 30 handle_osd_map epochs [31,31], i have 30, src has [1,31] 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 31 epoch_requested: 31 MQuery 2.0 from 3 query_epoch 31 query: query(info 0'0 epoch_sent 31)) class_id 2immediate prio 255 cost 10 e31) queued 2026-02-11T23:30:20.880+0000 7fac606e4640 10 osd.0 30 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:20.880+0000 7fac606e4640 10 osd.0 30 handle_osd_map got inc map for epoch 31 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 31 epoch_requested: 31 MQuery 2.0 from 3 query_epoch 31 query: query(info 0'0 epoch_sent 31)) class_id 2immediate prio 255 cost 10 e31) pg 0 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 map 31 > 30 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 2.0 peering, item epoch is 31, will wait on OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 31 epoch_requested: 31 MQuery 2.0 from 3 query_epoch 31 query: query(info 0'0 epoch_sent 31)) class_id 2immediate prio 255 cost 10 e31) 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.880+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.880+0000 7fac606e4640 20 osd.0 30 got_full_map 31, nothing requested 2026-02-11T23:30:20.880+0000 7fac606e4640 10 osd.0 30 _track_pools_and_pg_num_changes recording new pool 2 pg_num 1 2026-02-11T23:30:20.880+0000 7fac606e4640 20 osd.0 30 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:20.880+0000 7fac606e4640 20 osd.0 30 handle_osd_map pg_num_history pg_num_history(e31 pg_nums {1={24=1},2={31=1}} deleted_pools ) 2026-02-11T23:30:20.880+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {31={}} 2026-02-11T23:30:20.880+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:20.880+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e31 maps [1~31] lci=[5,31] tlb=1) 2026-02-11T23:30:20.880+0000 7fac5e983640 10 osd.0 30 _committed_osd_maps 31..31 2026-02-11T23:30:20.880+0000 7fac5e983640 10 osd.0 30 advance to epoch 31 (<= last 31 <= newest_map 31) 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0 31 consume_map version 31 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0 30 await_reserved_maps epoch:31 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0 30 await_reserved_maps done 0 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0 31 consume_map 31 -- publish done 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:2.0slot:0x2581600 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 2.0 has no pg and waiting_for_split 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0 31 identify_splits_and_merges 1.0 e30 to e31 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0 31 identify_splits_and_merges 1.0 e30 to e31 pg_nums {24=1} 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0 31 prune_pg_created 2026-02-11T23:30:20.880+0000 7fac5e983640 10 osd.0:0.consume_map 31 (was 30) 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 2.0 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 2.0 pending_peering first epoch 31 <= 31, requeueing 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 2.0 to_process <> waiting <> waiting_peering {31=} 2026-02-11T23:30:20.880+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:20.880+0000 7fac5e983640 10 osd.0:1.consume_map 31 (was 30) 2026-02-11T23:30:20.880+0000 7fac5e983640 10 osd.0:2.consume_map 31 (was 30) 2026-02-11T23:30:20.880+0000 7fac5e983640 10 osd.0:3.consume_map 31 (was 30) 2026-02-11T23:30:20.880+0000 7fac5e983640 10 osd.0:4.consume_map 31 (was 30) 2026-02-11T23:30:20.880+0000 7fac5e983640 10 osd.0:5.consume_map 31 (was 30) 2026-02-11T23:30:20.880+0000 7fac5e983640 10 osd.0:6.consume_map 31 (was 30) 2026-02-11T23:30:20.880+0000 7fac5e983640 10 osd.0:7.consume_map 31 (was 30) 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 31 epoch_requested: 31 MQuery 2.0 from 3 query_epoch 31 query: query(info 0'0 epoch_sent 31)) class_id 2immediate prio 255 cost 10 e31) queued 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 31 epoch_requested: 31 MQuery 2.0 from 3 query_epoch 31 query: query(info 0'0 epoch_sent 31)) class_id 2immediate prio 255 cost 10 e31) pg 0 2026-02-11T23:30:20.880+0000 7fac51168640 10 osd.0 31 handle_pg_query_nopg 2.0 2026-02-11T23:30:20.880+0000 7fac51168640 10 osd.0 31 pg 2.0 dne 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 31 get_con_osd_cluster to osd.3 from_epoch 31 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 31 get_nextmap_reserved map_reservations: {31=1} 2026-02-11T23:30:20.880+0000 7fac51168640 20 osd.0 31 release_map epoch: 31 2026-02-11T23:30:20.880+0000 7fac51168640 10 osd.0 31 maybe_share_map: con v2:127.0.0.1:6828/1114810440 map epoch 30 -> 31 (shared) 2026-02-11T23:30:20.880+0000 7fac51168640 10 osd.0 31 send_incremental_map epoch range: (31, 31] to v2:127.0.0.1:6828/1114810440 2026-02-11T23:30:20.881+0000 7fac51168640 10 osd.0 31 add_map_inc_bl 31 591 bytes 2026-02-11T23:30:20.881+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- osd_map(31..31 src has 1..31) -- 0x46ea000 con 0x3741000 2026-02-11T23:30:20.881+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_notify2(2.0 2.0 (query:31 sent:31 2.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e31/31) -- 0x3383800 con 0x3741000 2026-02-11T23:30:20.881+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.881+0000 7fac5e983640 10 osd.0 31 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:20.881+0000 7fac5e983640 15 osd.0 31 enqueue_peering_evt 1.0 epoch_sent: 31 epoch_requested: 31 NullEvt 2026-02-11T23:30:20.881+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 31 epoch_requested: 31 NullEvt) class_id 2 prio 255 cost 10 e31) 2026-02-11T23:30:20.881+0000 7fac5e983640 7 osd.0 31 activate_map version 31 2026-02-11T23:30:20.881+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.881+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 31 epoch_requested: 31 NullEvt) class_id 2immediate prio 255 cost 10 e31) queued 2026-02-11T23:30:20.881+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.881+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 31 epoch_requested: 31 NullEvt) class_id 2immediate prio 255 cost 10 e31) pg 0x40f8000 2026-02-11T23:30:20.881+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.881+0000 7fac55971640 20 osd.0 31 agent_entry empty queue 2026-02-11T23:30:20.881+0000 7fac51168640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 31 2026-02-11T23:30:20.881+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 30 -> 31 2026-02-11T23:30:20.881+0000 7fac51168640 10 osd.0 pg_epoch: 30 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:20.881+0000 7fac51168640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:20.881+0000 7fac51168640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:20.881+0000 7fac51168640 20 osd.0 31 advance_pg new pool opts old pool opts 2026-02-11T23:30:20.881+0000 7fac51168640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 31..31 2026-02-11T23:30:20.881+0000 7fac51168640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:20.881+0000 7fac51168640 20 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 31 2026-02-11T23:30:20.881+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:20.881+0000 7fac51168640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 31 epoch_requested: 31 NullEvt 2026-02-11T23:30:20.881+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:20.881+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.894+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 3 ==== PGlog(2.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.419069290s int 16.000000000s) e32/31) ==== 1172+0+0 (unknown 0 0 0) 0x3902000 con 0x3741000 2026-02-11T23:30:20.894+0000 7fac723fe640 15 osd.0 31 enqueue_peering_evt 2.0 epoch_sent: 32 epoch_requested: 31 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.419069290s int 16.000000000s) +create_info 2026-02-11T23:30:20.894+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 31 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.419069290s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e32) 2026-02-11T23:30:20.894+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:20.894+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 31 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.419069290s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e32) queued 2026-02-11T23:30:20.894+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:20.894+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 31 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.419069290s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e32) pg 0 2026-02-11T23:30:20.894+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 map 32 > 31 2026-02-11T23:30:20.894+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 2.0 peering, item epoch is 32, will wait on OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 31 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.419069290s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e32) 2026-02-11T23:30:20.894+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:20.894+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.081+0000 7fac68beb640 10 osd.0 31 tick 2026-02-11T23:30:21.081+0000 7fac68beb640 20 osd.0 31 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:21.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:21.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:21.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:51.168109+0000) 2026-02-11T23:30:21.266+0000 7fac683ea640 10 osd.0 31 tick_without_osd_lock 2026-02-11T23:30:21.266+0000 7fac683ea640 0 osd.0 31 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:21.266+0000 7fac683ea640 20 osd.0 31 tick_without_osd_lock max_waiting_epoch 32, requesting new map 2026-02-11T23:30:21.266+0000 7fac683ea640 10 monclient: _renew_subs 2026-02-11T23:30:21.266+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:21.266+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=32}) -- 0x38f5ba0 con 0x396bc00 2026-02-11T23:30:21.266+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:21.266+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.166 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:21.266+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:21.266, recovery is active?:false restrictions:<....> 2026-02-11T23:30:21.266+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:21.266+0000 7fac683ea640 10 osd.0 31 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:21.266+0000 7fac683ea640 20 osd.0 31 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:21.266+0000 7fac683ea640 10 osd.0 31 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:21.266+0000 7fac683ea640 20 osd.0 31 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:21.266+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 30 ==== osd_map(32..32 src has 1..32) ==== 296+0+0 (crc 0 0 0) 0x46ea000 con 0x396bc00 2026-02-11T23:30:21.266+0000 7fac626e8640 20 osd.0 31 OSD::ms_dispatch: osd_map(32..32 src has 1..32) 2026-02-11T23:30:21.266+0000 7fac626e8640 20 osd.0 31 _dispatch 0x46ea000 osd_map(32..32 src has 1..32) 2026-02-11T23:30:21.266+0000 7fac626e8640 3 osd.0 31 handle_osd_map epochs [32,32], i have 31, src has [1,32] 2026-02-11T23:30:21.266+0000 7fac626e8640 10 osd.0 31 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:21.266+0000 7fac626e8640 10 osd.0 31 handle_osd_map got inc map for epoch 32 2026-02-11T23:30:21.266+0000 7fac626e8640 20 osd.0 31 got_full_map 32, nothing requested 2026-02-11T23:30:21.266+0000 7fac626e8640 20 osd.0 31 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:21.266+0000 7fac626e8640 20 osd.0 31 handle_osd_map pg_num_history pg_num_history(e32 pg_nums {1={24=1},2={31=1}} deleted_pools ) 2026-02-11T23:30:21.266+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {32={}} 2026-02-11T23:30:21.266+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:21.266+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e32 maps [1~32] lci=[5,32] tlb=1) 2026-02-11T23:30:21.266+0000 7fac5e983640 10 osd.0 31 _committed_osd_maps 32..32 2026-02-11T23:30:21.266+0000 7fac5e983640 10 osd.0 31 advance to epoch 32 (<= last 32 <= newest_map 32) 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0 32 consume_map version 32 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0 31 await_reserved_maps epoch:32 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0 31 await_reserved_maps done 0 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0 32 consume_map 32 -- publish done 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:2.0slot:0x2581600 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 2.0 has no pg and waiting_for_split 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0 32 identify_splits_and_merges 1.0 e31 to e32 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0 32 identify_splits_and_merges 1.0 e31 to e32 pg_nums {24=1} 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0 32 prune_pg_created 2026-02-11T23:30:21.266+0000 7fac5e983640 10 osd.0:0.consume_map 32 (was 31) 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 2.0 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 2.0 pending_peering first epoch 32 <= 32, requeueing 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 2.0 to_process <> waiting <> waiting_peering {32=} 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:21.266+0000 7fac5e983640 10 osd.0:1.consume_map 32 (was 31) 2026-02-11T23:30:21.266+0000 7fac5e983640 10 osd.0:2.consume_map 32 (was 31) 2026-02-11T23:30:21.266+0000 7fac5e983640 10 osd.0:3.consume_map 32 (was 31) 2026-02-11T23:30:21.266+0000 7fac5e983640 10 osd.0:4.consume_map 32 (was 31) 2026-02-11T23:30:21.266+0000 7fac5e983640 10 osd.0:5.consume_map 32 (was 31) 2026-02-11T23:30:21.266+0000 7fac5e983640 10 osd.0:6.consume_map 32 (was 31) 2026-02-11T23:30:21.266+0000 7fac5e983640 10 osd.0:7.consume_map 32 (was 31) 2026-02-11T23:30:21.266+0000 7fac5e983640 10 osd.0 32 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 31 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.419069290s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e32) queued 2026-02-11T23:30:21.266+0000 7fac5e983640 15 osd.0 32 enqueue_peering_evt 1.0 epoch_sent: 32 epoch_requested: 32 NullEvt 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 31 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.419069290s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e32) pg 0 2026-02-11T23:30:21.266+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 NullEvt) class_id 2 prio 255 cost 10 e32) 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 no pg, should create on OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 31 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.419069290s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e32) 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 32 _make_pg 2.0 2026-02-11T23:30:21.266+0000 7fac55170640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:2, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:30:21.266+0000 7fac55170640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:2, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:30:21.266+0000 7fac55170640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:2, shard:-1, shard_prefix: , prefixes: {"0000000000000002."}] update_bits prefix updated 2026-02-11T23:30:21.266+0000 7fac55170640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:2, shard:-1, shard_prefix: , prefixes: {"0000000000000002."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:30:21.266+0000 7fac55170640 5 osd.0 pg_epoch: 31 pg[2.0(unlocked)] enter Initial 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 pg_epoch: 31 pg[2.0(unlocked)] enter NotTrimming 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role 2 up [3,1,0] acting [3,1,0] history ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31 past_intervals ([0,0] all_participants={} intervals=[]) 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [3,1,0]+[3,1,0] 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x38637a0] 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 14.101510048s + 0.000000000s) 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:21.266+0000 7fac55170640 15 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:21.266+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:21.266+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:21.266+0000 7fac55170640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:30:21.266+0000 7fac55170640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:30:21.266+0000 7fac55170640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:30:21.266+0000 7fac55170640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:30:21.266+0000 7fac55170640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:30:21.266+0000 7fac55170640 10 end of _write_log_and_missing 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:30:21.266+0000 7fac55170640 5 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] exit Initial 0.000082 0 0.000000 2026-02-11T23:30:21.266+0000 7fac55170640 5 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 31 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 31..31 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:21.266+0000 7fac55170640 5 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] exit Reset 0.000012 1 0.000012 2026-02-11T23:30:21.266+0000 7fac55170640 5 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:30:21.266+0000 7fac55170640 5 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:30:21.266+0000 7fac55170640 1 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:30:21.266+0000 7fac55170640 5 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] exit Start 0.000003 0 0.000000 2026-02-11T23:30:21.266+0000 7fac55170640 5 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 31 while current is 31 2026-02-11T23:30:21.266+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 32 handle_pg_create_info new pg pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0:0._attach_pg 2.0 0x46f4000 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0:0._wake_pg_slot _wake_pg_slot 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 32 identify_splits_and_merges 2.0 e31 to e32 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 32 identify_splits_and_merges 2.0 e31 to e32 pg_nums {31=1} 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0:0._prime_splits 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] handle_advance_map: 32 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0:0.update_pg_epoch 2.0 31 -> 32 2026-02-11T23:30:21.266+0000 7fac5e983640 7 osd.0 32 activate_map version 32 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 31 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [3,1,0]/[3,1,0] -- 3/3 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:21.266+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.266+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 NullEvt) class_id 2immediate prio 255 cost 10 e32) queued 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:21.266+0000 7fac55971640 20 osd.0 32 agent_entry empty queue 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 32 advance_pg new pool opts old pool opts 2026-02-11T23:30:21.266+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 32..32 2026-02-11T23:30:21.266+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 NullEvt) class_id 2immediate prio 255 cost 10 e32) pg 0x40f8000 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 31 while current is 32 2026-02-11T23:30:21.266+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 32 epoch_requested: 31 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 22.419069290s int 16.000000000s) +create_info 2026-02-11T23:30:21.266+0000 7fac51168640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 32 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] state: got info+log from osd.3 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:21.266+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 31 -> 32 2026-02-11T23:30:21.266+0000 7fac51168640 10 osd.0 pg_epoch: 31 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:21.266+0000 7fac55170640 10 merge_log log((0'0,0'0], crt=0'0) from osd.3 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:21.266+0000 7fac55170640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:21.266+0000 7fac51168640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:21.266+0000 7fac55170640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:21.266+0000 7fac55170640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:21.266+0000 7fac51168640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:21.266+0000 7fac51168640 20 osd.0 32 advance_pg new pool opts old pool opts 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 22.419069290s int 16.000000000s) 2026-02-11T23:30:21.266+0000 7fac51168640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 32..32 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] PeeringState::proc_lease peer_clock_delta_lb -7.310860634s -> ruub 29.729928970s 2026-02-11T23:30:21.266+0000 7fac51168640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] PeeringState::proc_lease readable_until_ub now 29.729928970s 2026-02-11T23:30:21.266+0000 7fac51168640 20 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 32 2026-02-11T23:30:21.266+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:21.266+0000 7fac55170640 5 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] exit Started/Stray 0.000081 3 0.000014 2026-02-11T23:30:21.266+0000 7fac51168640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 32 epoch_requested: 32 NullEvt 2026-02-11T23:30:21.266+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:21.266+0000 7fac55170640 5 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:21.266+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.266+0000 7fac55170640 5 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=0/0 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:21.266+0000 7fac55170640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:21.266+0000 7fac55170640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:21.266+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:30:21.266+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:21.266+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac55170640 15 osd.0 32 enqueue_peering_evt 2.0 epoch_sent: 32 epoch_requested: 32 ActivateCommitted from 32 processed at 32 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 ActivateCommitted from 32 processed at 32) class_id 2 prio 255 cost 10 e32) 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 ActivateCommitted from 32 processed at 32) class_id 2immediate prio 255 cost 10 e32) queued 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 ActivateCommitted from 32 processed at 32) class_id 2immediate prio 255 cost 10 e32) pg 0x46f4000 2026-02-11T23:30:21.267+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 32 epoch_requested: 32 ActivateCommitted from 32 processed at 32 2026-02-11T23:30:21.267+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 unknown mbc={}] state: react 32 telling primary 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 32 get_con_osd_cluster to osd.3 from_epoch 32 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 32 get_nextmap_reserved map_reservations: {32=1} 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 32 release_map epoch: 32 2026-02-11T23:30:21.267+0000 7fac55170640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6828/1114810440 map epoch 31 -> 32 (shared) 2026-02-11T23:30:21.267+0000 7fac55170640 10 osd.0 32 send_incremental_map epoch range: (32, 32] to v2:127.0.0.1:6828/1114810440 2026-02-11T23:30:21.267+0000 7fac55170640 10 osd.0 32 add_map_inc_bl 32 252 bytes 2026-02-11T23:30:21.267+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- osd_map(32..32 src has 1..32) -- 0x46ea700 con 0x3741000 2026-02-11T23:30:21.267+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_info2(2.0 2.0 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/0 les/c/f=32/0/0 sis=31) e32/32) -- 0x3382300 con 0x3741000 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 4 ==== pg_lease(2.0 pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s) e32/32) ==== 58+0+0 (unknown 0 0 0) 0x3919980 con 0x3741000 2026-02-11T23:30:21.267+0000 7fac723fe640 15 osd.0 32 enqueue_peering_evt 2.0 epoch_sent: 32 epoch_requested: 32 MLease epoch 32 from osd.3 pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s) 2026-02-11T23:30:21.267+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 MLease epoch 32 from osd.3 pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s)) class_id 2 prio 255 cost 10 e32) 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 MLease epoch 32 from osd.3 pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e32) queued 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 MLease epoch 32 from osd.3 pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e32) pg 0x46f4000 2026-02-11T23:30:21.267+0000 7fac51168640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 32 epoch_requested: 32 MLease epoch 32 from osd.3 pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s) 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac51168640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s) 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -7.310327530s -> ru 29.729394913s 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 29.729394913s 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -7.310860634s -> ruub 30.102554321s 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 30.102554321s 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 32 get_con_osd_cluster to osd.3 from_epoch 32 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 32 get_nextmap_reserved map_reservations: {32=1} 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 32 release_map epoch: 32 2026-02-11T23:30:21.267+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_lease_ack(2.0 pg_lease_ack(ruub 22.791692734s) e32/32) -- 0x2580000 con 0x3741000 2026-02-11T23:30:21.267+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 5 ==== pg_info2(2.0 2.0 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/0 les/c/f=32/0/0 sis=31) e32/32) ==== 1126+0+0 (unknown 0 0 0) 0x3382300 con 0x3741000 2026-02-11T23:30:21.267+0000 7fac723fe640 15 osd.0 32 enqueue_peering_evt 2.0 epoch_sent: 32 epoch_requested: 32 MInfoRec from 3 info: 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/0 les/c/f=32/0/0 sis=31) pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s) 2026-02-11T23:30:21.267+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 MInfoRec from 3 info: 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/0 les/c/f=32/0/0 sis=31) pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s)) class_id 2 prio 255 cost 10 e32) 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 MInfoRec from 3 info: 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/0 les/c/f=32/0/0 sis=31) pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e32) queued 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 MInfoRec from 3 info: 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/0 les/c/f=32/0/0 sis=31) pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e32) pg 0x46f4000 2026-02-11T23:30:21.267+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=0/0 les/c/f=0/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 32 epoch_requested: 32 MInfoRec from 3 info: 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/0 les/c/f=32/0/0 sis=31) pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s) 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/0 les/c/f=32/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=31/31 lis/c=31/0 les/c/f=32/0/0 sis=31 2026-02-11T23:30:21.267+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 6 ==== pg_info2(2.0 2.0 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) e32/32) ==== 1126+0+0 (unknown 0 0 0) 0x3382300 con 0x3741000 2026-02-11T23:30:21.267+0000 7fac723fe640 15 osd.0 32 enqueue_peering_evt 2.0 epoch_sent: 32 epoch_requested: 32 MInfoRec from 3 info: 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s) 2026-02-11T23:30:21.267+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 MInfoRec from 3 info: 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s)) class_id 2 prio 255 cost 10 e32) 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 MInfoRec from 3 info: 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e32) queued 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 32 epoch_requested: 32 MInfoRec from 3 info: 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e32) pg 0x46f4000 2026-02-11T23:30:21.267+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/0 les/c/f=32/0/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 32 epoch_requested: 32 MInfoRec from 3 info: 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) pg_lease(ru 22.419069290s ub 22.791692734s int 16.000000000s) 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:21.267+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.267+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.669+0000 7fac4d160640 5 osd.0 32 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x162bc/0x40000, compress 0x0/0x0/0x0, omap 0x3935, meta 0x1a4c6cb), peers [1,2,3,4] op hist []) 2026-02-11T23:30:21.669+0000 7fac4d160640 20 osd.0 32 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:21.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e32 up_from 5 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 14.503632545s delta_ub 1.981160998s) -- 0x3b1af00 con 0x396b400 2026-02-11T23:30:21.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e32 up_from 5 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 14.503632545s delta_ub 1.981160998s) -- 0x3b1a900 con 0x3752400 2026-02-11T23:30:21.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e32 up_from 5 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 14.503632545s delta_ub 4.649120808s) -- 0x3b1b200 con 0x396b800 2026-02-11T23:30:21.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e32 up_from 5 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 14.503632545s delta_ub 4.649120808s) -- 0x3b1b380 con 0x34c1000 2026-02-11T23:30:21.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e32 up_from 5 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 14.503632545s delta_ub 7.310860634s) -- 0x3b1b500 con 0x3740000 2026-02-11T23:30:21.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e32 up_from 5 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 14.503632545s delta_ub 7.310860634s) -- 0x3b1ad80 con 0x3740400 2026-02-11T23:30:21.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e32 up_from 5 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 14.503632545s delta_ub 9.887025833s) -- 0x3b1a000 con 0x3740800 2026-02-11T23:30:21.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e32 up_from 5 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 14.503632545s delta_ub 9.887025833s) -- 0x3b1aa80 con 0x3740c00 2026-02-11T23:30:21.669+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 4 ==== osd_ping(ping_reply e32 up_from 9 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 12.522871017s delta_ub -1.980761290s) ==== 2033+0+0 (crc 0 0 0) 0x3b1aa80 con 0x396b400 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 get_con_osd_cluster to osd.1 from_epoch 32 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 get_nextmap_reserved map_reservations: {32=1} 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 release_map epoch: 32 2026-02-11T23:30:21.669+0000 7fac703fa640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 28 to ping map epoch of 32 2026-02-11T23:30:21.669+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 1 ==== osd_ping(ping_reply e32 up_from 17 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 7.193384647s delta_ub -7.310247898s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b200 con 0x3740400 2026-02-11T23:30:21.669+0000 7fac703fa640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 32 is not newer than session's projected_epoch of 32 2026-02-11T23:30:21.669+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 4 ==== osd_ping(ping_reply e32 up_from 9 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 12.522902489s delta_ub -1.980729818s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x3752400 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980931878s,-1.980761290s]) 2026-02-11T23:30:21.669+0000 7fac71bfd640 20 osd.0 32 get_con_osd_cluster to osd.1 from_epoch 32 2026-02-11T23:30:21.669+0000 7fac71bfd640 20 osd.0 32 get_nextmap_reserved map_reservations: {32=1} 2026-02-11T23:30:21.669+0000 7fac71bfd640 20 osd.0 32 release_map epoch: 32 2026-02-11T23:30:21.669+0000 7fac71bfd640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 32 is not newer than session's projected_epoch of 32 2026-02-11T23:30:21.669+0000 7fac71bfd640 20 osd.0 32 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980933070s,-1.980729818s]) 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 get_con_osd_cluster to osd.3 from_epoch 32 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 get_nextmap_reserved map_reservations: {32=1} 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 release_map epoch: 32 2026-02-11T23:30:21.669+0000 7fac723fe640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 32 is not newer than session's projected_epoch of 32 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310467720s,-7.310247898s]) 2026-02-11T23:30:21.669+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 1 ==== osd_ping(ping_reply e28 up_from 21 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 4.616877556s delta_ub -9.886754990s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b200 con 0x3740800 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 get_con_osd_cluster to osd.4 from_epoch 32 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 get_nextmap_reserved map_reservations: {32=1} 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 release_map epoch: 32 2026-02-11T23:30:21.669+0000 7fac723fe640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 0 to ping map epoch of 28 2026-02-11T23:30:21.669+0000 7fac723fe640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 28 -> 32 (shared) 2026-02-11T23:30:21.669+0000 7fac723fe640 10 osd.0 32 send_incremental_map epoch range: (29, 32] to v2:127.0.0.1:6836/864727417 2026-02-11T23:30:21.669+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(29..32 src has 1..32) -- 0x46ea700 con 0x3e4a000 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886991501s,-9.886754990s]) 2026-02-11T23:30:21.669+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 1 ==== osd_ping(ping_reply e32 up_from 17 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 7.193409443s delta_ub -7.310222626s) ==== 2033+0+0 (crc 0 0 0) 0x3b1aa80 con 0x3740000 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 get_con_osd_cluster to osd.3 from_epoch 32 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 get_nextmap_reserved map_reservations: {32=1} 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 release_map epoch: 32 2026-02-11T23:30:21.669+0000 7fac703fa640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 32 is not newer than session's projected_epoch of 32 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310494423s,-7.310222626s]) 2026-02-11T23:30:21.669+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 1 ==== osd_ping(ping_reply e28 up_from 21 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 4.616911888s delta_ub -9.886720657s) ==== 2033+0+0 (crc 0 0 0) 0x3b1aa80 con 0x3740c00 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 get_con_osd_cluster to osd.4 from_epoch 32 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 get_nextmap_reserved map_reservations: {32=1} 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 release_map epoch: 32 2026-02-11T23:30:21.669+0000 7fac703fa640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 32 is not newer than session's projected_epoch of 32 2026-02-11T23:30:21.669+0000 7fac703fa640 20 osd.0 32 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887063980s,-9.886720657s]) 2026-02-11T23:30:21.669+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 3 ==== osd_ping(ping_reply e33 up_from 13 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 9.854722023s delta_ub -4.648910522s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b200 con 0x396b800 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 get_con_osd_cluster to osd.2 from_epoch 32 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 get_nextmap_reserved map_reservations: {32=1} 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 release_map epoch: 32 2026-02-11T23:30:21.669+0000 7fac723fe640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 21 to ping map epoch of 33 2026-02-11T23:30:21.669+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.2 v2:127.0.0.1:6820/2662569638 2 ==== osd_map(33..33 src has 1..33) ==== 618+0+0 (unknown 0 0 0) 0x39416c0 con 0x3f57c00 2026-02-11T23:30:21.669+0000 7fac723fe640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 32 is not newer than session's projected_epoch of 33 2026-02-11T23:30:21.669+0000 7fac606e4640 20 osd.0 32 OSD::ms_dispatch: osd_map(33..33 src has 1..33) 2026-02-11T23:30:21.669+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 3 ==== osd_ping(ping_reply e33 up_from 13 ping_stamp 2026-02-11T23:30:21.669217+0000/14.503632545s send_stamp 9.854910851s delta_ub -4.648722649s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x34c1000 2026-02-11T23:30:21.669+0000 7fac723fe640 20 osd.0 32 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649266720s,-4.648910522s]) 2026-02-11T23:30:21.669+0000 7fac606e4640 20 osd.0 32 _dispatch 0x39416c0 osd_map(33..33 src has 1..33) 2026-02-11T23:30:21.669+0000 7fac71bfd640 20 osd.0 32 get_con_osd_cluster to osd.2 from_epoch 32 2026-02-11T23:30:21.669+0000 7fac71bfd640 20 osd.0 32 get_nextmap_reserved map_reservations: {32=1} 2026-02-11T23:30:21.669+0000 7fac71bfd640 20 osd.0 32 release_map epoch: 32 2026-02-11T23:30:21.669+0000 7fac71bfd640 10 osd.0 32 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 32 is not newer than session's projected_epoch of 33 2026-02-11T23:30:21.669+0000 7fac71bfd640 20 osd.0 32 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649101257s,-4.648722649s]) 2026-02-11T23:30:21.669+0000 7fac606e4640 3 osd.0 32 handle_osd_map epochs [33,33], i have 32, src has [1,33] 2026-02-11T23:30:21.669+0000 7fac606e4640 10 osd.0 32 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:21.669+0000 7fac606e4640 10 osd.0 32 handle_osd_map got inc map for epoch 33 2026-02-11T23:30:21.669+0000 7fac606e4640 20 osd.0 32 got_full_map 33, nothing requested 2026-02-11T23:30:21.669+0000 7fac606e4640 20 osd.0 32 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:21.669+0000 7fac606e4640 20 osd.0 32 handle_osd_map pg_num_history pg_num_history(e33 pg_nums {1={24=1},2={31=1}} deleted_pools ) 2026-02-11T23:30:21.669+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {33={}} 2026-02-11T23:30:21.669+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:21.669+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e33 maps [1~33] lci=[5,33] tlb=1) 2026-02-11T23:30:21.669+0000 7fac5e983640 10 osd.0 32 _committed_osd_maps 33..33 2026-02-11T23:30:21.669+0000 7fac5e983640 10 osd.0 32 advance to epoch 33 (<= last 33 <= newest_map 33) 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0 33 consume_map version 33 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0 32 await_reserved_maps epoch:33 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0 32 await_reserved_maps done 0 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0 33 consume_map 33 -- publish done 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:2.0slot:0x2581600 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0 33 identify_splits_and_merges 2.0 e32 to e33 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0 33 identify_splits_and_merges 2.0 e32 to e33 pg_nums {31=1} 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0 33 identify_splits_and_merges 1.0 e32 to e33 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0 33 identify_splits_and_merges 1.0 e32 to e33 pg_nums {24=1} 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0 33 prune_pg_created 2026-02-11T23:30:21.669+0000 7fac5e983640 10 osd.0:0.consume_map 33 (was 32) 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 2.0 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:21.669+0000 7fac5e983640 10 osd.0:1.consume_map 33 (was 32) 2026-02-11T23:30:21.669+0000 7fac5e983640 10 osd.0:2.consume_map 33 (was 32) 2026-02-11T23:30:21.669+0000 7fac5e983640 10 osd.0:3.consume_map 33 (was 32) 2026-02-11T23:30:21.669+0000 7fac5e983640 10 osd.0:4.consume_map 33 (was 32) 2026-02-11T23:30:21.669+0000 7fac5e983640 10 osd.0:5.consume_map 33 (was 32) 2026-02-11T23:30:21.669+0000 7fac5e983640 10 osd.0:6.consume_map 33 (was 32) 2026-02-11T23:30:21.669+0000 7fac5e983640 10 osd.0:7.consume_map 33 (was 32) 2026-02-11T23:30:21.669+0000 7fac5e983640 10 osd.0 33 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:21.669+0000 7fac5e983640 15 osd.0 33 enqueue_peering_evt 2.0 epoch_sent: 33 epoch_requested: 33 NullEvt 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 33 epoch_requested: 33 NullEvt) class_id 2 prio 255 cost 10 e33) 2026-02-11T23:30:21.669+0000 7fac5e983640 15 osd.0 33 enqueue_peering_evt 1.0 epoch_sent: 33 epoch_requested: 33 NullEvt 2026-02-11T23:30:21.669+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 33 epoch_requested: 33 NullEvt) class_id 2 prio 255 cost 10 e33) 2026-02-11T23:30:21.669+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.669+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 33 epoch_requested: 33 NullEvt) class_id 2immediate prio 255 cost 10 e33) queued 2026-02-11T23:30:21.669+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:21.669+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 33 epoch_requested: 33 NullEvt) class_id 2immediate prio 255 cost 10 e33) pg 0x46f4000 2026-02-11T23:30:21.669+0000 7fac5e983640 7 osd.0 33 activate_map version 33 2026-02-11T23:30:21.669+0000 7fac55971640 20 osd.0 33 agent_entry empty queue 2026-02-11T23:30:21.669+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] handle_advance_map: 33 2026-02-11T23:30:21.669+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.669+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 33 epoch_requested: 33 NullEvt) class_id 2immediate prio 255 cost 10 e33) queued 2026-02-11T23:30:21.669+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:21.669+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 33 epoch_requested: 33 NullEvt) class_id 2immediate prio 255 cost 10 e33) pg 0x40f8000 2026-02-11T23:30:21.669+0000 7fac55170640 20 osd.0:0.update_pg_epoch 2.0 32 -> 33 2026-02-11T23:30:21.669+0000 7fac55170640 10 osd.0 pg_epoch: 32 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [3,1,0]/[3,1,0] -- 3/3 2026-02-11T23:30:21.669+0000 7fac51168640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 33 2026-02-11T23:30:21.669+0000 7fac55170640 10 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:21.669+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 32 -> 33 2026-02-11T23:30:21.669+0000 7fac55170640 10 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:21.669+0000 7fac51168640 10 osd.0 pg_epoch: 32 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:21.669+0000 7fac55170640 10 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:21.669+0000 7fac51168640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:21.669+0000 7fac55170640 20 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:21.669+0000 7fac51168640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:21.669+0000 7fac55170640 20 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:21.669+0000 7fac51168640 20 osd.0 33 advance_pg new pool opts old pool opts 2026-02-11T23:30:21.669+0000 7fac55170640 20 osd.0 33 advance_pg new pool opts old pool opts 2026-02-11T23:30:21.669+0000 7fac51168640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 33..33 2026-02-11T23:30:21.669+0000 7fac55170640 10 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] handle_activate_map: epoch range: 33..33 2026-02-11T23:30:21.669+0000 7fac51168640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:21.669+0000 7fac55170640 10 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:21.669+0000 7fac51168640 20 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 33 2026-02-11T23:30:21.669+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:21.669+0000 7fac55170640 20 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 32 while current is 33 2026-02-11T23:30:21.669+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:21.669+0000 7fac51168640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 33 epoch_requested: 33 NullEvt 2026-02-11T23:30:21.669+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:21.669+0000 7fac55170640 10 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:21.669+0000 7fac55170640 10 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 33 epoch_requested: 33 NullEvt 2026-02-11T23:30:21.669+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.669+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:21.669+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.976+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 31 ==== osd_map(33..34 src has 1..34) ==== 1200+0+0 (crc 0 0 0) 0x3941180 con 0x396bc00 2026-02-11T23:30:21.976+0000 7fac626e8640 20 osd.0 33 OSD::ms_dispatch: osd_map(33..34 src has 1..34) 2026-02-11T23:30:21.976+0000 7fac626e8640 20 osd.0 33 _dispatch 0x3941180 osd_map(33..34 src has 1..34) 2026-02-11T23:30:21.976+0000 7fac626e8640 3 osd.0 33 handle_osd_map epochs [33,34], i have 33, src has [1,34] 2026-02-11T23:30:21.976+0000 7fac626e8640 10 osd.0 33 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:21.976+0000 7fac626e8640 10 osd.0 33 handle_osd_map got inc map for epoch 34 2026-02-11T23:30:21.976+0000 7fac626e8640 20 osd.0 33 got_full_map 34, nothing requested 2026-02-11T23:30:21.976+0000 7fac626e8640 20 osd.0 33 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:21.976+0000 7fac626e8640 20 osd.0 33 handle_osd_map pg_num_history pg_num_history(e34 pg_nums {1={24=1},2={31=1}} deleted_pools ) 2026-02-11T23:30:21.976+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {34={}} 2026-02-11T23:30:21.976+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:21.976+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e34 maps [1~34] lci=[5,34] tlb=1) 2026-02-11T23:30:21.977+0000 7fac5e983640 10 osd.0 33 _committed_osd_maps 34..34 2026-02-11T23:30:21.977+0000 7fac5e983640 10 osd.0 33 advance to epoch 34 (<= last 34 <= newest_map 34) 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0 34 consume_map version 34 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0 33 await_reserved_maps epoch:34 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0 33 await_reserved_maps done 0 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0 34 consume_map 34 -- publish done 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:2.0slot:0x2581600 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0 34 identify_splits_and_merges 2.0 e33 to e34 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0 34 identify_splits_and_merges 2.0 e33 to e34 pg_nums {31=1} 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0 34 identify_splits_and_merges 1.0 e33 to e34 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0 34 identify_splits_and_merges 1.0 e33 to e34 pg_nums {24=1} 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0 34 prune_pg_created 2026-02-11T23:30:21.977+0000 7fac5e983640 10 osd.0:0.consume_map 34 (was 33) 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 2.0 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:21.977+0000 7fac5e983640 10 osd.0:1.consume_map 34 (was 33) 2026-02-11T23:30:21.977+0000 7fac5e983640 10 osd.0:2.consume_map 34 (was 33) 2026-02-11T23:30:21.977+0000 7fac5e983640 10 osd.0:3.consume_map 34 (was 33) 2026-02-11T23:30:21.977+0000 7fac5e983640 10 osd.0:4.consume_map 34 (was 33) 2026-02-11T23:30:21.977+0000 7fac5e983640 10 osd.0:5.consume_map 34 (was 33) 2026-02-11T23:30:21.977+0000 7fac5e983640 10 osd.0:6.consume_map 34 (was 33) 2026-02-11T23:30:21.977+0000 7fac5e983640 10 osd.0:7.consume_map 34 (was 33) 2026-02-11T23:30:21.977+0000 7fac5e983640 10 osd.0 34 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:21.977+0000 7fac5e983640 15 osd.0 34 enqueue_peering_evt 2.0 epoch_sent: 34 epoch_requested: 34 NullEvt 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 NullEvt) class_id 2 prio 255 cost 10 e34) 2026-02-11T23:30:21.977+0000 7fac5e983640 15 osd.0 34 enqueue_peering_evt 1.0 epoch_sent: 34 epoch_requested: 34 NullEvt 2026-02-11T23:30:21.977+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 NullEvt) class_id 2 prio 255 cost 10 e34) 2026-02-11T23:30:21.977+0000 7fac5e983640 7 osd.0 34 activate_map version 34 2026-02-11T23:30:21.977+0000 7fac55971640 20 osd.0 34 agent_entry empty queue 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 NullEvt) class_id 2immediate prio 255 cost 10 e34) queued 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 NullEvt) class_id 2immediate prio 255 cost 10 e34) pg 0x46f4000 2026-02-11T23:30:21.977+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.977+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 NullEvt) class_id 2immediate prio 255 cost 10 e34) queued 2026-02-11T23:30:21.977+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:21.977+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 NullEvt) class_id 2immediate prio 255 cost 10 e34) pg 0x40f8000 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] handle_advance_map: 34 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0:0.update_pg_epoch 2.0 33 -> 34 2026-02-11T23:30:21.977+0000 7fac55170640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 34 2026-02-11T23:30:21.977+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 33 -> 34 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 33 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [3,1]/[3,1] -- 3/3 2026-02-11T23:30:21.977+0000 7fac55170640 10 osd.0 pg_epoch: 33 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:21.977+0000 7fac55170640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [3,1] newacting [3,1] 2026-02-11T23:30:21.977+0000 7fac55170640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:21.977+0000 7fac51168640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.710472 8 0.000058 2026-02-11T23:30:21.977+0000 7fac55170640 20 osd.0 34 advance_pg new pool opts old pool opts 2026-02-11T23:30:21.977+0000 7fac51168640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] exit Started/ReplicaActive 0.710478 0 0.000000 2026-02-11T23:30:21.977+0000 7fac55170640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 34..34 2026-02-11T23:30:21.977+0000 7fac55170640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:21.977+0000 7fac51168640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] exit Started 0.710573 0 0.000000 2026-02-11T23:30:21.977+0000 7fac55170640 20 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 34 2026-02-11T23:30:21.977+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:21.977+0000 7fac51168640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:21.977+0000 7fac55170640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 34 epoch_requested: 34 NullEvt 2026-02-11T23:30:21.977+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=31 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 34 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:21.977+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [3,1] newacting [3,1] 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1,0] r=2 lpr=34 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 34 2026-02-11T23:30:21.977+0000 7fac51168640 15 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1] r=2 lpr=34 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(31-33 up [3, 1, 0](3) acting [3, 1, 0](3)) up_thru 31 up_from 17 last_epoch_clean 32 interval(31-33 up [3, 1, 0](3) acting [3, 1, 0](3) maybe_went_rw) : primary up 17-31 includes interval 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([31,33] all_participants={0, 1, 3} intervals=[([31,33] acting={0, 1, 3})]) 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [3,1]+[3,1] 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active pruub 30.102554321s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 30.102554321s (mnow 14.812197685s + 15.290356636s) 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active pruub 30.102554321s@ mbc={}] cancel_recovery 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active pruub 30.102554321s@ mbc={}] clear_recovery_state 2026-02-11T23:30:21.977+0000 7fac51168640 15 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active pruub 30.102554321s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active pruub 30.102554321s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:21.977+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active pruub 30.102554321s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active pruub 30.102554321s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active pruub 30.102554321s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:21.977+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:21.977+0000 7fac51168640 1 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 active pruub 30.102554321s@ mbc={}] PeeringState::start_peering_interval up [3,1,0] -> [3,1], acting [3,1,0] -> [3,1], acting_primary 3 -> 3, up_primary 3 -> 3, role 2 -> -1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] clear_primary_state 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] release_backoffs [2:00000000::::0,MAX) 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:21.977+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:21.977+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] agent_stop 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] on_change 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] on_change_cleanup 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] on_change 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] exit NotTrimming 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown pruub 30.102554321s@ mbc={}] enter NotTrimming 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] plpg_on_role_change 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] hit_set_clear 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] agent_stop 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 34 advance_pg new pool opts old pool opts 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] handle_activate_map: epoch range: 34..34 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290356636s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:21.977+0000 7fac51168640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290221214s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] exit Reset 0.000215 1 0.000212 2026-02-11T23:30:21.977+0000 7fac51168640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290221214s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] enter Started 2026-02-11T23:30:21.977+0000 7fac51168640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290221214s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] enter Start 2026-02-11T23:30:21.977+0000 7fac51168640 1 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290221214s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:21.977+0000 7fac51168640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290221214s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] exit Start 0.000006 0 0.000000 2026-02-11T23:30:21.977+0000 7fac51168640 5 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290221214s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] enter Started/Stray 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290221214s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 32 while current is 34 2026-02-11T23:30:21.977+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290221214s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290221214s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] do_peering_event: epoch_sent: 34 epoch_requested: 34 NullEvt 2026-02-11T23:30:21.977+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 34 get_con_osd_cluster to osd.3 from_epoch 34 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6828/1114810440 map epoch 32 -> 34 (shared) 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 34 send_incremental_map epoch range: (33, 34] to v2:127.0.0.1:6828/1114810440 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 34 add_map_inc_bl 33 574 bytes 2026-02-11T23:30:21.977+0000 7fac51168640 10 osd.0 34 add_map_inc_bl 34 574 bytes 2026-02-11T23:30:21.977+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- osd_map(33..34 src has 1..34) -- 0x46ea1c0 con 0x3741000 2026-02-11T23:30:21.977+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_notify2(2.0 2.0 (query:34 sent:34 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290221214s) ([31,33] all_participants={0, 1, 3} intervals=[([31,33] acting={0, 1, 3})])) e34/34) -- 0x3382a00 con 0x3741000 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.977+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.977+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.977+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.978+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 7 ==== pg_query2(2.0 2.0 query(info 0'0 epoch_sent 34) e34/34) ==== 158+0+0 (unknown 0 0 0) 0x3f3f0e0 con 0x3741000 2026-02-11T23:30:21.978+0000 7fac723fe640 15 osd.0 34 enqueue_peering_evt 2.0 epoch_sent: 34 epoch_requested: 34 MQuery 2.0 from 3 query_epoch 34 query: query(info 0'0 epoch_sent 34) 2026-02-11T23:30:21.978+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 MQuery 2.0 from 3 query_epoch 34 query: query(info 0'0 epoch_sent 34)) class_id 2 prio 255 cost 10 e34) 2026-02-11T23:30:21.978+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:21.978+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 MQuery 2.0 from 3 query_epoch 34 query: query(info 0'0 epoch_sent 34)) class_id 2immediate prio 255 cost 10 e34) queued 2026-02-11T23:30:21.978+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:21.978+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 34 epoch_requested: 34 MQuery 2.0 from 3 query_epoch 34 query: query(info 0'0 epoch_sent 34)) class_id 2immediate prio 255 cost 10 e34) pg 0x46f4000 2026-02-11T23:30:21.978+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.290221214s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] do_peering_event: epoch_sent: 34 epoch_requested: 34 MQuery 2.0 from 3 query_epoch 34 query: query(info 0'0 epoch_sent 34) 2026-02-11T23:30:21.978+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.978+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102554321s@ mbc={}] PeeringState::update_history advanced history from ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s 2026-02-11T23:30:21.978+0000 7fac51168640 20 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] PeeringState::update_history prior_readable_until_ub 30.102384567s (mnow 14.813654900s + 15.288730621s) 2026-02-11T23:30:21.978+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 8 ==== osd_map(33..34 src has 1..34) ==== 1200+0+0 (unknown 0 0 0) 0x46ea1c0 con 0x3741000 2026-02-11T23:30:21.978+0000 7fac51168640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:21.978+0000 7fac606e4640 20 osd.0 34 OSD::ms_dispatch: osd_map(33..34 src has 1..34) 2026-02-11T23:30:21.978+0000 7fac606e4640 20 osd.0 34 _dispatch 0x46ea1c0 osd_map(33..34 src has 1..34) 2026-02-11T23:30:21.978+0000 7fac606e4640 3 osd.0 34 handle_osd_map epochs [33,34], i have 34, src has [1,34] 2026-02-11T23:30:21.978+0000 7fac606e4640 10 osd.0 34 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:21.978+0000 7fac606e4640 10 osd.0 34 no new maps here, dropping 2026-02-11T23:30:21.978+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:21.978+0000 7fac51168640 20 osd.0 34 get_con_osd_cluster to osd.3 from_epoch 34 2026-02-11T23:30:21.978+0000 7fac51168640 20 osd.0 34 get_nextmap_reserved map_reservations: {34=1} 2026-02-11T23:30:21.978+0000 7fac51168640 20 osd.0 34 release_map epoch: 34 2026-02-11T23:30:21.978+0000 7fac51168640 10 osd.0 34 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 34 is not newer than session's projected_epoch of 34 2026-02-11T23:30:21.978+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_notify2(2.0 2.0 (query:34 sent:34 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) ([31,33] all_participants={0, 1, 3} intervals=[([31,33] acting={0, 1, 3})])) e34/34) -- 0x3383100 con 0x3741000 2026-02-11T23:30:21.978+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.978+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:21.978+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.053+0000 7fac68beb640 10 osd.0 34 tick 2026-02-11T23:30:22.053+0000 7fac68beb640 20 osd.0 34 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:22.090+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 9 ==== osd pg remove(epoch 35; pg2.0; ) ==== 32+0+0 (unknown 0 0 0) 0x2580000 con 0x3741000 2026-02-11T23:30:22.090+0000 7fac723fe640 7 osd.0 34 handle_fast_pg_remove osd pg remove(epoch 35; pg2.0; ) from osd.3 2026-02-11T23:30:22.090+0000 7fac723fe640 15 osd.0 34 enqueue_peering_evt 2.0 epoch_sent: 35 epoch_requested: 35 DeleteStart 2026-02-11T23:30:22.090+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 DeleteStart) class_id 2 prio 255 cost 10 e35) 2026-02-11T23:30:22.090+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:22.090+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 DeleteStart) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:22.090+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:22.090+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 DeleteStart) class_id 2immediate prio 255 cost 10 e35) pg 0x46f4000 2026-02-11T23:30:22.090+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 2.0 peering, item epoch is 35, will wait on OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 DeleteStart) class_id 2immediate prio 255 cost 10 e35) 2026-02-11T23:30:22.090+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.090+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:22.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:22.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:52.168165+0000) 2026-02-11T23:30:22.272+0000 7fac683ea640 10 osd.0 34 tick_without_osd_lock 2026-02-11T23:30:22.272+0000 7fac683ea640 0 osd.0 34 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:22.272+0000 7fac683ea640 20 osd.0 34 tick_without_osd_lock max_waiting_epoch 35, requesting new map 2026-02-11T23:30:22.272+0000 7fac683ea640 10 monclient: _renew_subs 2026-02-11T23:30:22.272+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:22.272+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=35}) -- 0x44a6820 con 0x396bc00 2026-02-11T23:30:22.272+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:22.272+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:22.272+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:22.272+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.166 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:22.272+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:22.272, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:22.272+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:22.272+0000 7fac683ea640 10 osd.0 34 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:22.272+0000 7fac683ea640 20 osd.0 34 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:22.272+0000 7fac683ea640 10 osd.0 34 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:22.272+0000 7fac683ea640 20 osd.0 34 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:22.272+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 32 ==== osd_map(35..35 src has 1..35) ==== 626+0+0 (crc 0 0 0) 0x3941dc0 con 0x396bc00 2026-02-11T23:30:22.272+0000 7fac626e8640 20 osd.0 34 OSD::ms_dispatch: osd_map(35..35 src has 1..35) 2026-02-11T23:30:22.272+0000 7fac626e8640 20 osd.0 34 _dispatch 0x3941dc0 osd_map(35..35 src has 1..35) 2026-02-11T23:30:22.272+0000 7fac626e8640 3 osd.0 34 handle_osd_map epochs [35,35], i have 34, src has [1,35] 2026-02-11T23:30:22.272+0000 7fac626e8640 10 osd.0 34 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:22.272+0000 7fac626e8640 10 osd.0 34 handle_osd_map got inc map for epoch 35 2026-02-11T23:30:22.272+0000 7fac626e8640 20 osd.0 34 got_full_map 35, nothing requested 2026-02-11T23:30:22.272+0000 7fac626e8640 20 osd.0 34 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:22.272+0000 7fac626e8640 20 osd.0 34 handle_osd_map pg_num_history pg_num_history(e35 pg_nums {1={24=1},2={31=1}} deleted_pools ) 2026-02-11T23:30:22.272+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {35={}} 2026-02-11T23:30:22.272+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:22.272+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e35 maps [1~35] lci=[5,35] tlb=1) 2026-02-11T23:30:22.273+0000 7fac5e983640 10 osd.0 34 _committed_osd_maps 35..35 2026-02-11T23:30:22.273+0000 7fac5e983640 10 osd.0 34 advance to epoch 35 (<= last 35 <= newest_map 35) 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0 35 consume_map version 35 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0 34 await_reserved_maps epoch:35 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0 34 await_reserved_maps done 0 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0 35 consume_map 35 -- publish done 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:2.0slot:0x2581600 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0 35 identify_splits_and_merges 2.0 e34 to e35 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0 35 identify_splits_and_merges 2.0 e34 to e35 pg_nums {31=1} 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0 35 identify_splits_and_merges 1.0 e34 to e35 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0 35 identify_splits_and_merges 1.0 e34 to e35 pg_nums {24=1} 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0 35 prune_pg_created 2026-02-11T23:30:22.273+0000 7fac5e983640 10 osd.0:0.consume_map 35 (was 34) 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 2.0 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 2.0 pending_peering first epoch 35 <= 35, requeueing 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 2.0 to_process <> waiting <> waiting_peering {35=} 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:22.273+0000 7fac5e983640 10 osd.0:1.consume_map 35 (was 34) 2026-02-11T23:30:22.273+0000 7fac5e983640 10 osd.0:2.consume_map 35 (was 34) 2026-02-11T23:30:22.273+0000 7fac5e983640 10 osd.0:3.consume_map 35 (was 34) 2026-02-11T23:30:22.273+0000 7fac5e983640 10 osd.0:4.consume_map 35 (was 34) 2026-02-11T23:30:22.273+0000 7fac5e983640 10 osd.0:5.consume_map 35 (was 34) 2026-02-11T23:30:22.273+0000 7fac5e983640 10 osd.0:6.consume_map 35 (was 34) 2026-02-11T23:30:22.273+0000 7fac5e983640 10 osd.0:7.consume_map 35 (was 34) 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 DeleteStart) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 DeleteStart) class_id 2immediate prio 255 cost 10 e35) pg 0x46f4000 2026-02-11T23:30:22.273+0000 7fac5e983640 10 osd.0 35 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:22.273+0000 7fac5e983640 15 osd.0 35 enqueue_peering_evt 2.0 epoch_sent: 35 epoch_requested: 35 NullEvt 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] handle_advance_map: 35 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0:0.update_pg_epoch 2.0 34 -> 35 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 NullEvt) class_id 2 prio 255 cost 10 e35) 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 34 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] PeeringState::advance_map handle_advance_map [3,1]/[3,1] -- 3/3 2026-02-11T23:30:22.273+0000 7fac5e983640 15 osd.0 35 enqueue_peering_evt 1.0 epoch_sent: 35 epoch_requested: 35 NullEvt 2026-02-11T23:30:22.273+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 NullEvt) class_id 2 prio 255 cost 10 e35) 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] state: Started advmap 2026-02-11T23:30:22.273+0000 7fac5e983640 7 osd.0 35 activate_map version 35 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:22.273+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:22.273+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 NullEvt) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] hit_set_clear 2026-02-11T23:30:22.273+0000 7fac55971640 20 osd.0 35 agent_entry empty queue 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] agent_stop 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 35 advance_pg new pool opts old pool opts 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] handle_activate_map: epoch range: 35..35 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=15.288730621s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 34 while current is 35 2026-02-11T23:30:22.273+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] do_peering_event: epoch_sent: 35 epoch_requested: 35 DeleteStart 2026-02-11T23:30:22.273+0000 7fac55170640 5 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] exit Started/Stray 0.295963 5 0.000066 2026-02-11T23:30:22.273+0000 7fac55170640 5 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] enter Started/ToDelete 2026-02-11T23:30:22.273+0000 7fac55170640 5 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] enter Started/ToDelete/WaitDeleteReserved 2026-02-11T23:30:22.273+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 35 get_con_osd_cluster to osd.3 from_epoch 35 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 35 get_nextmap_reserved map_reservations: {35=1} 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 35 release_map epoch: 35 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 35 maybe_share_map: con v2:127.0.0.1:6828/1114810440 map epoch 34 -> 35 (shared) 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 35 send_incremental_map epoch range: (35, 35] to v2:127.0.0.1:6828/1114810440 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 35 add_map_inc_bl 35 582 bytes 2026-02-11T23:30:22.273+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- osd_map(35..35 src has 1..35) -- 0x46ea8c0 con 0x3741000 2026-02-11T23:30:22.273+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_notify2(2.0 2.0 (query:35 sent:35 2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) ([31,33] all_participants={0, 1, 3} intervals=[([31,33] acting={0, 1, 3})])) e35/35) -- 0x3383800 con 0x3741000 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 NullEvt) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 NullEvt) class_id 2immediate prio 255 cost 10 e35) pg 0x40f8000 2026-02-11T23:30:22.273+0000 7fac51168640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:22.273+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 NullEvt) class_id 2immediate prio 255 cost 10 e35) pg 0x46f4000 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 35 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 34 -> 35 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 34 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:22.273+0000 7fac51168640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] do_peering_event: epoch_sent: 35 epoch_requested: 35 NullEvt 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:22.273+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 35 advance_pg new pool opts old pool opts 2026-02-11T23:30:22.273+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 35..35 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 35 2026-02-11T23:30:22.273+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 35 epoch_requested: 35 NullEvt 2026-02-11T23:30:22.273+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.273+0000 7fac58176640 15 osd.0 35 enqueue_peering_evt 2.0 epoch_sent: 35 epoch_requested: 35 DeleteReserved 2026-02-11T23:30:22.273+0000 7fac58176640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 DeleteReserved) class_id 2 prio 255 cost 10 e35) 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 DeleteReserved) class_id 2immediate prio 255 cost 10 e35) queued 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGPeeringEvent(epoch_sent: 35 epoch_requested: 35 DeleteReserved) class_id 2immediate prio 255 cost 10 e35) pg 0x46f4000 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] do_peering_event: epoch_sent: 35 epoch_requested: 35 DeleteReserved 2026-02-11T23:30:22.273+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.273+0000 7fac55170640 5 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] exit Started/ToDelete/WaitDeleteReserved 0.000238 2 0.000021 2026-02-11T23:30:22.273+0000 7fac55170640 5 osd.0 pg_epoch: 35 pg[2.0( empty local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] enter Started/ToDelete/Deleting 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] on_removal 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] on_shutdown 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:22.273+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:22.273+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] clear_backoffs 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] exit NotTrimming 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] enter NotTrimming 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] on_change 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] clear_async_reads 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] clear_primary_state 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] release_backoffs [2:00000000::::0,MAX) 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:22.273+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:22.273+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] agent_stop 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] cancel_recovery 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] clear_recovery_state 2026-02-11T23:30:22.273+0000 7fac55170640 15 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:22.273+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.273+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 35 queue_for_pg_delete on 2.0 e 35 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(2.0 PGDelete(pgid=2.0 epoch_queued=35) class_id 1 prio 5 cost 1048576 e35) 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGDelete(2.0 e35) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e35) queued 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process 2.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:22.273+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(2.0 PGDelete(2.0 e35) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e35) pg 0x46f4000 2026-02-11T23:30:22.273+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] do_peering_event: epoch_sent: 35 epoch_requested: 35 DeleteSome 2026-02-11T23:30:22.273+0000 7fac55170640 10 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] do_delete_work 2026-02-11T23:30:22.274+0000 7fac55170640 20 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] do_delete_work [#2:00000000::::head#] 2026-02-11T23:30:22.274+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.274+0000 7fac55170640 20 osd.0 35 try_finish_pg_delete 2.0 0x46f4000 2026-02-11T23:30:22.274+0000 7fac55170640 10 osd.0:0._detach_pg 2.0 0x46f4000 2026-02-11T23:30:22.274+0000 7fac55170640 5 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 DELETING pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] exit Started/ToDelete/Deleting 0.000341 1 0.000070 2026-02-11T23:30:22.274+0000 7fac55170640 5 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] exit Started/ToDelete 0.000589 0 0.000000 2026-02-11T23:30:22.274+0000 7fac55170640 5 osd.0 pg_epoch: 35 pg[2.0( empty lb MIN local-lis/les=31/32 n=0 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=34 pruub=14.994083405s) [3,1] r=-1 lpr=34 pi=[31,34)/1 crt=0'0 unknown NOTIFY pruub 30.102384567s@ mbc={}] exit Started 0.296572 0 0.000000 2026-02-11T23:30:22.274+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:22.274+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.274+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.274+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:22.274+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:22.274+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:22.549+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 4 ==== osd_ping(ping e35 up_from 9 ping_stamp 2026-02-11T23:30:22.549452+0000/13.403017044s send_stamp 13.403017044s delta_ub -1.980729818s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x396a400 2026-02-11T23:30:22.549+0000 7fac71bfd640 20 osd.0 35 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980939150s,-1.980729818s]) 2026-02-11T23:30:22.549+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 4 ==== osd_ping(ping e35 up_from 9 ping_stamp 2026-02-11T23:30:22.549452+0000/13.403017044s send_stamp 13.403017044s delta_ub -1.980729818s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b200 con 0x396a000 2026-02-11T23:30:22.549+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e35 up_from 5 ping_stamp 2026-02-11T23:30:22.549452+0000/13.403017044s send_stamp 15.383956909s delta_ub 1.980939150s) -- 0x3b1a900 con 0x396a400 2026-02-11T23:30:22.549+0000 7fac71bfd640 20 osd.0 35 get_con_osd_cluster to osd.1 from_epoch 35 2026-02-11T23:30:22.549+0000 7fac71bfd640 20 osd.0 35 get_nextmap_reserved map_reservations: {35=1} 2026-02-11T23:30:22.549+0000 7fac71bfd640 20 osd.0 35 release_map epoch: 35 2026-02-11T23:30:22.549+0000 7fac71bfd640 10 osd.0 35 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 32 to ping map epoch of 35 2026-02-11T23:30:22.549+0000 7fac71bfd640 10 osd.0 35 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 35 is not newer than session's projected_epoch of 35 2026-02-11T23:30:22.549+0000 7fac723fe640 20 osd.0 35 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980966330s,-1.980729818s]) 2026-02-11T23:30:22.549+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e35 up_from 5 ping_stamp 2026-02-11T23:30:22.549452+0000/13.403017044s send_stamp 15.383982658s delta_ub 1.980966330s) -- 0x3b1ad80 con 0x396a000 2026-02-11T23:30:22.549+0000 7fac723fe640 20 osd.0 35 get_con_osd_cluster to osd.1 from_epoch 35 2026-02-11T23:30:22.549+0000 7fac723fe640 20 osd.0 35 get_nextmap_reserved map_reservations: {35=1} 2026-02-11T23:30:22.549+0000 7fac723fe640 20 osd.0 35 release_map epoch: 35 2026-02-11T23:30:22.549+0000 7fac723fe640 10 osd.0 35 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 35 is not newer than session's projected_epoch of 35 2026-02-11T23:30:23.010+0000 7fac60ee5640 20 osd.0 35 reports for 0 queries 2026-02-11T23:30:23.010+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x389d180 con 0x396a800 2026-02-11T23:30:23.010+0000 7fac60ee5640 15 osd.0 35 collect_pg_stats 2026-02-11T23:30:23.010+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836484 v 0) -- 0x3e48900 con 0x396a800 2026-02-11T23:30:23.061+0000 7fac68beb640 10 osd.0 35 tick 2026-02-11T23:30:23.061+0000 7fac68beb640 20 osd.0 35 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:23.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:23.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:23.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:53.168226+0000) 2026-02-11T23:30:23.314+0000 7fac683ea640 10 osd.0 35 tick_without_osd_lock 2026-02-11T23:30:23.314+0000 7fac683ea640 0 osd.0 35 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:23.314+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:23.314+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:23.314+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:23.314+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.166 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:23.314+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:23.315, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:23.314+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:23.314+0000 7fac683ea640 10 osd.0 35 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:23.314+0000 7fac683ea640 20 osd.0 35 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:23.314+0000 7fac683ea640 10 osd.0 35 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:23.314+0000 7fac683ea640 20 osd.0 35 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:23.454+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 2 ==== osd_ping(ping e39 up_from 21 ping_stamp 2026-02-11T23:30:23.455346+0000/6.402853012s send_stamp 6.402853012s delta_ub -9.886720657s) ==== 2033+0+0 (crc 0 0 0) 0x3b1aa80 con 0x3f57400 2026-02-11T23:30:23.454+0000 7fac703fa640 20 osd.0 35 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887005806s,-9.886720657s]) 2026-02-11T23:30:23.454+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e35 up_from 5 ping_stamp 2026-02-11T23:30:23.455346+0000/6.402853012s send_stamp 16.289857864s delta_ub 9.887005806s) -- 0x3b1b500 con 0x3f57400 2026-02-11T23:30:23.454+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 2 ==== osd_ping(ping e39 up_from 21 ping_stamp 2026-02-11T23:30:23.455346+0000/6.402853012s send_stamp 6.402853012s delta_ub -9.886720657s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a900 con 0x3f57800 2026-02-11T23:30:23.454+0000 7fac703fa640 20 osd.0 35 get_con_osd_cluster to osd.4 from_epoch 35 2026-02-11T23:30:23.454+0000 7fac703fa640 20 osd.0 35 get_nextmap_reserved map_reservations: {35=1} 2026-02-11T23:30:23.454+0000 7fac703fa640 20 osd.0 35 release_map epoch: 35 2026-02-11T23:30:23.454+0000 7fac703fa640 10 osd.0 35 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 32 to ping map epoch of 39 2026-02-11T23:30:23.454+0000 7fac703fa640 10 osd.0 35 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 35 is not newer than session's projected_epoch of 39 2026-02-11T23:30:23.454+0000 7fac71bfd640 20 osd.0 35 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887043953s,-9.886720657s]) 2026-02-11T23:30:23.454+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e35 up_from 5 ping_stamp 2026-02-11T23:30:23.455346+0000/6.402853012s send_stamp 16.289897919s delta_ub 9.887043953s) -- 0x3b1b380 con 0x3f57800 2026-02-11T23:30:23.454+0000 7fac71bfd640 20 osd.0 35 get_con_osd_cluster to osd.4 from_epoch 35 2026-02-11T23:30:23.454+0000 7fac71bfd640 20 osd.0 35 get_nextmap_reserved map_reservations: {35=1} 2026-02-11T23:30:23.454+0000 7fac71bfd640 20 osd.0 35 release_map epoch: 35 2026-02-11T23:30:23.454+0000 7fac71bfd640 10 osd.0 35 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 35 is not newer than session's projected_epoch of 39 2026-02-11T23:30:23.454+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1 ==== osd_map(36..39 src has 1..39) ==== 2059+0+0 (unknown 0 0 0) 0x46ea700 con 0x3e4a000 2026-02-11T23:30:23.454+0000 7fac606e4640 20 osd.0 35 OSD::ms_dispatch: osd_map(36..39 src has 1..39) 2026-02-11T23:30:23.454+0000 7fac606e4640 20 osd.0 35 _dispatch 0x46ea700 osd_map(36..39 src has 1..39) 2026-02-11T23:30:23.454+0000 7fac606e4640 3 osd.0 35 handle_osd_map epochs [36,39], i have 35, src has [1,39] 2026-02-11T23:30:23.454+0000 7fac606e4640 10 osd.0 35 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:23.454+0000 7fac606e4640 10 osd.0 35 handle_osd_map got inc map for epoch 36 2026-02-11T23:30:23.454+0000 7fac606e4640 20 osd.0 35 got_full_map 36, nothing requested 2026-02-11T23:30:23.454+0000 7fac606e4640 10 osd.0 35 handle_osd_map got inc map for epoch 37 2026-02-11T23:30:23.454+0000 7fac606e4640 20 osd.0 35 got_full_map 37, nothing requested 2026-02-11T23:30:23.454+0000 7fac606e4640 10 osd.0 35 handle_osd_map got inc map for epoch 38 2026-02-11T23:30:23.454+0000 7fac606e4640 20 osd.0 35 got_full_map 38, nothing requested 2026-02-11T23:30:23.454+0000 7fac606e4640 10 osd.0 35 handle_osd_map got inc map for epoch 39 2026-02-11T23:30:23.454+0000 7fac606e4640 20 osd.0 35 got_full_map 39, nothing requested 2026-02-11T23:30:23.454+0000 7fac606e4640 10 osd.0 35 _track_pools_and_pg_num_changes recording new pool 3 pg_num 1 2026-02-11T23:30:23.454+0000 7fac606e4640 20 osd.0 35 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:23.454+0000 7fac606e4640 20 osd.0 35 handle_osd_map pg_num_history pg_num_history(e39 pg_nums {1={24=1},2={31=1},3={38=1}} deleted_pools ) 2026-02-11T23:30:23.454+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {36={},37={},38={},39={}} 2026-02-11T23:30:23.454+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:23.454+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e39 maps [1~39] lci=[5,39] tlb=1) 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0 35 _committed_osd_maps 36..39 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0 35 advance to epoch 36 (<= last 39 <= newest_map 39) 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0 36 advance to epoch 37 (<= last 39 <= newest_map 39) 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0 37 advance to epoch 38 (<= last 39 <= newest_map 39) 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0 38 advance to epoch 39 (<= last 39 <= newest_map 39) 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0 39 consume_map version 39 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0 35 await_reserved_maps epoch:39 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0 35 await_reserved_maps done 0 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0 39 consume_map 39 -- publish done 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:2.0slot:0x2581600 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 2.0 has no pg and waiting_for_split 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0 39 identify_splits_and_merges 1.0 e35 to e39 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0 39 identify_splits_and_merges 1.0 e35 to e39 pg_nums {24=1} 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0 39 prune_pg_created 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0:0.consume_map 39 (was 35) 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 2.0 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 2.0 empty, pruning 2026-02-11T23:30:23.454+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0:1.consume_map 39 (was 35) 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0:2.consume_map 39 (was 35) 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0:3.consume_map 39 (was 35) 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0:4.consume_map 39 (was 35) 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0:5.consume_map 39 (was 35) 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0:6.consume_map 39 (was 35) 2026-02-11T23:30:23.454+0000 7fac5e983640 10 osd.0:7.consume_map 39 (was 35) 2026-02-11T23:30:23.455+0000 7fac5e983640 10 osd.0 39 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:23.455+0000 7fac5e983640 15 osd.0 39 enqueue_peering_evt 1.0 epoch_sent: 39 epoch_requested: 39 NullEvt 2026-02-11T23:30:23.455+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 39 epoch_requested: 39 NullEvt) class_id 2 prio 255 cost 10 e39) 2026-02-11T23:30:23.455+0000 7fac5e983640 7 osd.0 39 activate_map version 39 2026-02-11T23:30:23.455+0000 7fac55971640 20 osd.0 39 agent_entry empty queue 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 39 epoch_requested: 39 NullEvt) class_id 2immediate prio 255 cost 10 e39) queued 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 39 epoch_requested: 39 NullEvt) class_id 2immediate prio 255 cost 10 e39) pg 0x40f8000 2026-02-11T23:30:23.455+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 36 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 35 -> 36 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 35 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 36 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 36 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0 39 advance_pg new pool opts old pool opts 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 36 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 37 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 36 -> 37 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 36 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 37 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 37 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0 39 advance_pg new pool opts old pool opts 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 37 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 38 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 37 -> 38 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 37 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0 39 advance_pg new pool opts old pool opts 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 39 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 38 -> 39 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 38 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0 39 advance_pg new pool opts old pool opts 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 36..39 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 39 2026-02-11T23:30:23.455+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:23.455+0000 7fac51168640 10 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 39 epoch_requested: 39 NullEvt 2026-02-11T23:30:23.455+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:23.455+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:23.954+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 3 ==== osd_ping(ping e39 up_from 21 ping_stamp 2026-02-11T23:30:23.955600+0000/6.903107643s send_stamp 6.903107643s delta_ub -9.886780739s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b500 con 0x3f57400 2026-02-11T23:30:23.954+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 3 ==== osd_ping(ping e39 up_from 21 ping_stamp 2026-02-11T23:30:23.955600+0000/6.903107643s send_stamp 6.903107643s delta_ub -9.886780739s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x3f57800 2026-02-11T23:30:23.954+0000 7fac703fa640 20 osd.0 39 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886999130s,-9.886780739s]) 2026-02-11T23:30:23.954+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e39 up_from 5 ping_stamp 2026-02-11T23:30:23.955600+0000/6.903107643s send_stamp 16.790105820s delta_ub 9.886999130s) -- 0x3b1aa80 con 0x3f57400 2026-02-11T23:30:23.954+0000 7fac703fa640 20 osd.0 39 get_con_osd_cluster to osd.4 from_epoch 39 2026-02-11T23:30:23.954+0000 7fac703fa640 20 osd.0 39 get_nextmap_reserved map_reservations: {39=1} 2026-02-11T23:30:23.954+0000 7fac703fa640 20 osd.0 39 release_map epoch: 39 2026-02-11T23:30:23.954+0000 7fac703fa640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 39 is not newer than session's projected_epoch of 39 2026-02-11T23:30:23.954+0000 7fac71bfd640 20 osd.0 39 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887020111s,-9.886780739s]) 2026-02-11T23:30:23.954+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e39 up_from 5 ping_stamp 2026-02-11T23:30:23.955600+0000/6.903107643s send_stamp 16.790126801s delta_ub 9.887020111s) -- 0x3b1a900 con 0x3f57800 2026-02-11T23:30:23.954+0000 7fac71bfd640 20 osd.0 39 get_con_osd_cluster to osd.4 from_epoch 39 2026-02-11T23:30:23.954+0000 7fac71bfd640 20 osd.0 39 get_nextmap_reserved map_reservations: {39=1} 2026-02-11T23:30:23.954+0000 7fac71bfd640 20 osd.0 39 release_map epoch: 39 2026-02-11T23:30:23.954+0000 7fac71bfd640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 39 is not newer than session's projected_epoch of 39 2026-02-11T23:30:23.968+0000 7fac4d160640 5 osd.0 39 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x1a3ee/0x48000, compress 0x0/0x0/0x0, omap 0x4361, meta 0x1a4bc9f), peers [1,2,3,4] op hist []) 2026-02-11T23:30:23.968+0000 7fac4d160640 20 osd.0 39 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:23.968+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e39 up_from 5 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 16.803861618s delta_ub 1.980966330s) -- 0x3b1a300 con 0x396b400 2026-02-11T23:30:23.968+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e39 up_from 5 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 16.803861618s delta_ub 1.980966330s) -- 0x3b1a600 con 0x3752400 2026-02-11T23:30:23.968+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e39 up_from 5 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 16.803861618s delta_ub 4.649101257s) -- 0x3b1b080 con 0x396b800 2026-02-11T23:30:23.968+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e39 up_from 5 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 16.803861618s delta_ub 4.649101257s) -- 0x3b1b980 con 0x34c1000 2026-02-11T23:30:23.968+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e39 up_from 5 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 16.803861618s delta_ub 7.310494423s) -- 0x3b1bb00 con 0x3740000 2026-02-11T23:30:23.968+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e39 up_from 5 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 16.803861618s delta_ub 7.310494423s) -- 0x3b1bc80 con 0x3740400 2026-02-11T23:30:23.968+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e39 up_from 5 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 16.803861618s delta_ub 9.887020111s) -- 0x3b1be00 con 0x3740800 2026-02-11T23:30:23.968+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e39 up_from 5 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 16.803861618s delta_ub 9.887020111s) -- 0x3b1b800 con 0x3740c00 2026-02-11T23:30:23.968+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 5 ==== osd_ping(ping_reply e41 up_from 9 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 14.823074341s delta_ub -1.980787992s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b980 con 0x3752400 2026-02-11T23:30:23.968+0000 7fac71bfd640 20 osd.0 39 get_con_osd_cluster to osd.1 from_epoch 39 2026-02-11T23:30:23.968+0000 7fac71bfd640 20 osd.0 39 get_nextmap_reserved map_reservations: {39=1} 2026-02-11T23:30:23.968+0000 7fac71bfd640 20 osd.0 39 release_map epoch: 39 2026-02-11T23:30:23.968+0000 7fac71bfd640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 35 to ping map epoch of 41 2026-02-11T23:30:23.968+0000 7fac71bfd640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 39 is not newer than session's projected_epoch of 41 2026-02-11T23:30:23.968+0000 7fac71bfd640 20 osd.0 39 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980907679s,-1.980787992s]) 2026-02-11T23:30:23.968+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 4 ==== osd_ping(ping_reply e41 up_from 13 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 12.154919624s delta_ub -4.648942947s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x396b800 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 get_con_osd_cluster to osd.2 from_epoch 39 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 get_nextmap_reserved map_reservations: {39=1} 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 release_map epoch: 39 2026-02-11T23:30:23.968+0000 7fac723fe640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 33 to ping map epoch of 41 2026-02-11T23:30:23.968+0000 7fac723fe640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 39 is not newer than session's projected_epoch of 41 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649088860s,-4.648942947s]) 2026-02-11T23:30:23.968+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 15 ==== osd_map(40..41 src has 1..41) ==== 1200+0+0 (unknown 0 0 0) 0x38e21c0 con 0x3752000 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 OSD::ms_dispatch: osd_map(40..41 src has 1..41) 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 _dispatch 0x38e21c0 osd_map(40..41 src has 1..41) 2026-02-11T23:30:23.968+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 2 ==== osd_ping(ping_reply e41 up_from 21 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 6.917077065s delta_ub -9.886785507s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x3740800 2026-02-11T23:30:23.968+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 5 ==== osd_ping(ping_reply e41 up_from 9 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 14.823101997s delta_ub -1.980759859s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b500 con 0x396b400 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 get_con_osd_cluster to osd.4 from_epoch 39 2026-02-11T23:30:23.968+0000 7fac606e4640 3 osd.0 39 handle_osd_map epochs [40,41], i have 39, src has [1,41] 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 get_nextmap_reserved map_reservations: {39=1} 2026-02-11T23:30:23.968+0000 7fac606e4640 10 osd.0 39 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 release_map epoch: 39 2026-02-11T23:30:23.968+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 4 ==== osd_ping(ping_reply e41 up_from 13 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 12.154953003s delta_ub -4.648910046s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b980 con 0x34c1000 2026-02-11T23:30:23.968+0000 7fac606e4640 10 osd.0 39 handle_osd_map got inc map for epoch 40 2026-02-11T23:30:23.968+0000 7fac723fe640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 39 to ping map epoch of 41 2026-02-11T23:30:23.968+0000 7fac723fe640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 39 is not newer than session's projected_epoch of 41 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886960983s,-9.886785507s]) 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 get_con_osd_cluster to osd.1 from_epoch 39 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 get_nextmap_reserved map_reservations: {39=1} 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 release_map epoch: 39 2026-02-11T23:30:23.968+0000 7fac703fa640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 39 is not newer than session's projected_epoch of 41 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980949044s,-1.980759859s]) 2026-02-11T23:30:23.968+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 2 ==== osd_ping(ping_reply e41 up_from 17 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 9.493624687s delta_ub -7.310237408s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x3740400 2026-02-11T23:30:23.968+0000 7fac71bfd640 20 osd.0 39 get_con_osd_cluster to osd.2 from_epoch 39 2026-02-11T23:30:23.968+0000 7fac71bfd640 20 osd.0 39 get_nextmap_reserved map_reservations: {39=1} 2026-02-11T23:30:23.968+0000 7fac71bfd640 20 osd.0 39 release_map epoch: 39 2026-02-11T23:30:23.968+0000 7fac71bfd640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 39 is not newer than session's projected_epoch of 41 2026-02-11T23:30:23.968+0000 7fac71bfd640 20 osd.0 39 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649113655s,-4.648910046s]) 2026-02-11T23:30:23.968+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 2 ==== osd_ping(ping_reply e41 up_from 17 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 9.493579865s delta_ub -7.310282230s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b500 con 0x3740000 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 get_con_osd_cluster to osd.3 from_epoch 39 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 get_nextmap_reserved map_reservations: {39=1} 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 release_map epoch: 39 2026-02-11T23:30:23.968+0000 7fac703fa640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 35 to ping map epoch of 41 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 got_full_map 40, nothing requested 2026-02-11T23:30:23.968+0000 7fac703fa640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 39 is not newer than session's projected_epoch of 41 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310495377s,-7.310282230s]) 2026-02-11T23:30:23.968+0000 7fac606e4640 10 osd.0 39 handle_osd_map got inc map for epoch 41 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 get_con_osd_cluster to osd.3 from_epoch 39 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 get_nextmap_reserved map_reservations: {39=1} 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 release_map epoch: 39 2026-02-11T23:30:23.968+0000 7fac723fe640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 39 is not newer than session's projected_epoch of 41 2026-02-11T23:30:23.968+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 2 ==== osd_ping(ping_reply e41 up_from 21 ping_stamp 2026-02-11T23:30:23.969446+0000/16.803861618s send_stamp 6.917104244s delta_ub -9.886757851s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b500 con 0x3740c00 2026-02-11T23:30:23.968+0000 7fac723fe640 20 osd.0 39 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310461044s,-7.310237408s]) 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 got_full_map 41, nothing requested 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 get_con_osd_cluster to osd.4 from_epoch 39 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 get_nextmap_reserved map_reservations: {39=1} 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 release_map epoch: 39 2026-02-11T23:30:23.968+0000 7fac703fa640 10 osd.0 39 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 39 is not newer than session's projected_epoch of 41 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:23.968+0000 7fac703fa640 20 osd.0 39 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886997223s,-9.886757851s]) 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 handle_osd_map pg_num_history pg_num_history(e41 pg_nums {1={24=1},2={31=1},3={38=1}} deleted_pools ) 2026-02-11T23:30:23.968+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {40={},41={}} 2026-02-11T23:30:23.968+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:23.968+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e41 maps [1~41] lci=[5,41] tlb=1) 2026-02-11T23:30:23.968+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.2 v2:127.0.0.1:6820/2662569638 3 ==== osd_map(40..41 src has 1..41) ==== 1200+0+0 (unknown 0 0 0) 0x3941880 con 0x3f57c00 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 OSD::ms_dispatch: osd_map(40..41 src has 1..41) 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 _dispatch 0x3941880 osd_map(40..41 src has 1..41) 2026-02-11T23:30:23.968+0000 7fac606e4640 3 osd.0 39 handle_osd_map epochs [40,41], i have 41, src has [1,41] 2026-02-11T23:30:23.968+0000 7fac606e4640 10 osd.0 39 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:23.968+0000 7fac606e4640 10 osd.0 39 no new maps here, dropping 2026-02-11T23:30:23.968+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 10 ==== osd_map(40..41 src has 1..41) ==== 1200+0+0 (unknown 0 0 0) 0x46ea8c0 con 0x3741000 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 OSD::ms_dispatch: osd_map(40..41 src has 1..41) 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 _dispatch 0x46ea8c0 osd_map(40..41 src has 1..41) 2026-02-11T23:30:23.968+0000 7fac606e4640 3 osd.0 39 handle_osd_map epochs [40,41], i have 41, src has [1,41] 2026-02-11T23:30:23.968+0000 7fac606e4640 10 osd.0 39 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:23.968+0000 7fac606e4640 10 osd.0 39 no new maps here, dropping 2026-02-11T23:30:23.968+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 2 ==== osd_map(40..41 src has 1..41) ==== 1200+0+0 (unknown 0 0 0) 0x3940700 con 0x3e4a000 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 OSD::ms_dispatch: osd_map(40..41 src has 1..41) 2026-02-11T23:30:23.968+0000 7fac606e4640 20 osd.0 39 _dispatch 0x3940700 osd_map(40..41 src has 1..41) 2026-02-11T23:30:23.968+0000 7fac606e4640 3 osd.0 39 handle_osd_map epochs [40,41], i have 41, src has [1,41] 2026-02-11T23:30:23.968+0000 7fac606e4640 10 osd.0 39 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:23.968+0000 7fac606e4640 10 osd.0 39 no new maps here, dropping 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0 39 _committed_osd_maps 40..41 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0 39 advance to epoch 40 (<= last 41 <= newest_map 41) 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0 40 advance to epoch 41 (<= last 41 <= newest_map 41) 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0 41 consume_map version 41 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0 39 await_reserved_maps epoch:41 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0 39 await_reserved_maps done 0 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0 41 consume_map 41 -- publish done 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0 41 identify_splits_and_merges 1.0 e39 to e41 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0 41 identify_splits_and_merges 1.0 e39 to e41 pg_nums {24=1} 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0 41 prune_pg_created 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0:0.consume_map 41 (was 39) 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0:1.consume_map 41 (was 39) 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0:2.consume_map 41 (was 39) 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0:3.consume_map 41 (was 39) 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0:4.consume_map 41 (was 39) 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0:5.consume_map 41 (was 39) 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0:6.consume_map 41 (was 39) 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0:7.consume_map 41 (was 39) 2026-02-11T23:30:23.968+0000 7fac5e983640 10 osd.0 41 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:23.968+0000 7fac5e983640 15 osd.0 41 enqueue_peering_evt 1.0 epoch_sent: 41 epoch_requested: 41 NullEvt 2026-02-11T23:30:23.968+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 41 epoch_requested: 41 NullEvt) class_id 2 prio 255 cost 10 e41) 2026-02-11T23:30:23.968+0000 7fac5e983640 7 osd.0 41 activate_map version 41 2026-02-11T23:30:23.968+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:23.968+0000 7fac55971640 20 osd.0 41 agent_entry empty queue 2026-02-11T23:30:23.968+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 41 epoch_requested: 41 NullEvt) class_id 2immediate prio 255 cost 10 e41) queued 2026-02-11T23:30:23.968+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:23.968+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 41 epoch_requested: 41 NullEvt) class_id 2immediate prio 255 cost 10 e41) pg 0x40f8000 2026-02-11T23:30:23.968+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:23.968+0000 7fac51168640 10 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 40 2026-02-11T23:30:23.968+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 39 -> 40 2026-02-11T23:30:23.968+0000 7fac51168640 10 osd.0 pg_epoch: 39 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:23.968+0000 7fac51168640 10 osd.0 pg_epoch: 40 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:23.968+0000 7fac51168640 10 osd.0 pg_epoch: 40 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:23.968+0000 7fac51168640 20 osd.0 41 advance_pg new pool opts old pool opts 2026-02-11T23:30:23.968+0000 7fac51168640 10 osd.0 pg_epoch: 40 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 41 2026-02-11T23:30:23.968+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 40 -> 41 2026-02-11T23:30:23.968+0000 7fac51168640 10 osd.0 pg_epoch: 40 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:23.968+0000 7fac51168640 10 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:23.968+0000 7fac51168640 10 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:23.968+0000 7fac51168640 20 osd.0 41 advance_pg new pool opts old pool opts 2026-02-11T23:30:23.968+0000 7fac51168640 10 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 40..41 2026-02-11T23:30:23.968+0000 7fac51168640 10 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:23.968+0000 7fac51168640 20 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 41 2026-02-11T23:30:23.968+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:23.968+0000 7fac51168640 10 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 41 epoch_requested: 41 NullEvt 2026-02-11T23:30:23.968+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:23.968+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:24.012+0000 7fac68beb640 10 osd.0 41 tick 2026-02-11T23:30:24.012+0000 7fac68beb640 20 osd.0 41 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:24.116+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 3 ==== osd_ping(ping e41 up_from 13 ping_stamp 2026-02-11T23:30:24.117633+0000/12.303028107s send_stamp 12.303028107s delta_ub -4.648910046s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x3f56000 2026-02-11T23:30:24.116+0000 7fac723fe640 20 osd.0 41 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649089336s,-4.648910046s]) 2026-02-11T23:30:24.116+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e41 up_from 5 ping_stamp 2026-02-11T23:30:24.117633+0000/12.303028107s send_stamp 16.952117920s delta_ub 4.649089336s) -- 0x3b1bb00 con 0x3f56000 2026-02-11T23:30:24.116+0000 7fac723fe640 20 osd.0 41 get_con_osd_cluster to osd.2 from_epoch 41 2026-02-11T23:30:24.116+0000 7fac723fe640 20 osd.0 41 get_nextmap_reserved map_reservations: {41=1} 2026-02-11T23:30:24.116+0000 7fac723fe640 20 osd.0 41 release_map epoch: 41 2026-02-11T23:30:24.116+0000 7fac723fe640 10 osd.0 41 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 41 is not newer than session's projected_epoch of 41 2026-02-11T23:30:24.116+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 3 ==== osd_ping(ping e41 up_from 13 ping_stamp 2026-02-11T23:30:24.117633+0000/12.303028107s send_stamp 12.303028107s delta_ub -4.648910046s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b500 con 0x3f56400 2026-02-11T23:30:24.116+0000 7fac703fa640 20 osd.0 41 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649135590s,-4.648910046s]) 2026-02-11T23:30:24.116+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e41 up_from 5 ping_stamp 2026-02-11T23:30:24.117633+0000/12.303028107s send_stamp 16.952163696s delta_ub 4.649135590s) -- 0x3b1aa80 con 0x3f56400 2026-02-11T23:30:24.116+0000 7fac703fa640 20 osd.0 41 get_con_osd_cluster to osd.2 from_epoch 41 2026-02-11T23:30:24.116+0000 7fac703fa640 20 osd.0 41 get_nextmap_reserved map_reservations: {41=1} 2026-02-11T23:30:24.116+0000 7fac703fa640 20 osd.0 41 release_map epoch: 41 2026-02-11T23:30:24.116+0000 7fac703fa640 10 osd.0 41 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 41 is not newer than session's projected_epoch of 41 2026-02-11T23:30:24.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:24.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:24.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:54.168275+0000) 2026-02-11T23:30:24.274+0000 7fac683ea640 10 osd.0 41 tick_without_osd_lock 2026-02-11T23:30:24.274+0000 7fac683ea640 0 osd.0 41 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:24.274+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:24.274+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:24.274+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:24.274+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.166 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:24.274+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:24.275, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:24.274+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:24.274+0000 7fac683ea640 10 osd.0 41 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:24.274+0000 7fac683ea640 20 osd.0 41 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:24.274+0000 7fac683ea640 10 osd.0 41 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:24.274+0000 7fac683ea640 20 osd.0 41 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:24.482+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 4 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 45) e45/45) ==== 158+0+0 (unknown 0 0 0) 0x3f3f680 con 0x3e4a000 2026-02-11T23:30:24.482+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 3 ==== osd_map(42..45 src has 1..45) ==== 2389+0+0 (unknown 0 0 0) 0x3941a40 con 0x3e4a000 2026-02-11T23:30:24.482+0000 7fac606e4640 20 osd.0 41 OSD::ms_dispatch: osd_map(42..45 src has 1..45) 2026-02-11T23:30:24.482+0000 7fac606e4640 20 osd.0 41 _dispatch 0x3941a40 osd_map(42..45 src has 1..45) 2026-02-11T23:30:24.482+0000 7fac703fa640 15 osd.0 41 enqueue_peering_evt 4.0 epoch_sent: 45 epoch_requested: 45 MQuery 4.0 from 4 query_epoch 45 query: query(info 0'0 epoch_sent 45) 2026-02-11T23:30:24.482+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 45 epoch_requested: 45 MQuery 4.0 from 4 query_epoch 45 query: query(info 0'0 epoch_sent 45)) class_id 2 prio 255 cost 10 e45) 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 45 epoch_requested: 45 MQuery 4.0 from 4 query_epoch 45 query: query(info 0'0 epoch_sent 45)) class_id 2immediate prio 255 cost 10 e45) queued 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 45 epoch_requested: 45 MQuery 4.0 from 4 query_epoch 45 query: query(info 0'0 epoch_sent 45)) class_id 2immediate prio 255 cost 10 e45) pg 0 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 map 45 > 41 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 45, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 45 epoch_requested: 45 MQuery 4.0 from 4 query_epoch 45 query: query(info 0'0 epoch_sent 45)) class_id 2immediate prio 255 cost 10 e45) 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:24.482+0000 7fac606e4640 3 osd.0 41 handle_osd_map epochs [42,45], i have 41, src has [1,45] 2026-02-11T23:30:24.482+0000 7fac606e4640 10 osd.0 41 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:24.482+0000 7fac606e4640 10 osd.0 41 handle_osd_map got inc map for epoch 42 2026-02-11T23:30:24.482+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:24.482+0000 7fac606e4640 20 osd.0 41 got_full_map 42, nothing requested 2026-02-11T23:30:24.482+0000 7fac606e4640 10 osd.0 41 handle_osd_map got inc map for epoch 43 2026-02-11T23:30:24.482+0000 7fac606e4640 20 osd.0 41 got_full_map 43, nothing requested 2026-02-11T23:30:24.482+0000 7fac606e4640 10 osd.0 41 handle_osd_map got inc map for epoch 44 2026-02-11T23:30:24.482+0000 7fac606e4640 20 osd.0 41 got_full_map 44, nothing requested 2026-02-11T23:30:24.482+0000 7fac606e4640 10 osd.0 41 handle_osd_map got inc map for epoch 45 2026-02-11T23:30:24.482+0000 7fac606e4640 20 osd.0 41 got_full_map 45, nothing requested 2026-02-11T23:30:24.482+0000 7fac606e4640 10 osd.0 41 _track_pools_and_pg_num_changes recording new pool 4 pg_num 1 2026-02-11T23:30:24.482+0000 7fac606e4640 20 osd.0 41 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:24.482+0000 7fac606e4640 20 osd.0 41 handle_osd_map pg_num_history pg_num_history(e45 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1}} deleted_pools ) 2026-02-11T23:30:24.482+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {42={},43={},44={},45={}} 2026-02-11T23:30:24.482+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:24.482+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e45 maps [1~45] lci=[5,45] tlb=1) 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0 41 _committed_osd_maps 42..45 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0 41 advance to epoch 42 (<= last 45 <= newest_map 45) 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0 42 advance to epoch 43 (<= last 45 <= newest_map 45) 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0 43 advance to epoch 44 (<= last 45 <= newest_map 45) 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0 44 advance to epoch 45 (<= last 45 <= newest_map 45) 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0 45 consume_map version 45 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0 41 await_reserved_maps epoch:45 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0 41 await_reserved_maps done 0 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0 45 consume_map 45 -- publish done 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 4.0 has no pg and waiting_for_split 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0 45 identify_splits_and_merges 1.0 e41 to e45 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0 45 identify_splits_and_merges 1.0 e41 to e45 pg_nums {24=1} 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0 45 prune_pg_created 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0:0.consume_map 45 (was 41) 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 45 <= 45, requeueing 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {45=} 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0:1.consume_map 45 (was 41) 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0:2.consume_map 45 (was 41) 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0:3.consume_map 45 (was 41) 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0:4.consume_map 45 (was 41) 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0:5.consume_map 45 (was 41) 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0:6.consume_map 45 (was 41) 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0:7.consume_map 45 (was 41) 2026-02-11T23:30:24.482+0000 7fac5e983640 10 osd.0 45 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:24.482+0000 7fac5e983640 15 osd.0 45 enqueue_peering_evt 1.0 epoch_sent: 45 epoch_requested: 45 NullEvt 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:24.482+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 45 epoch_requested: 45 NullEvt) class_id 2 prio 255 cost 10 e45) 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 45 epoch_requested: 45 MQuery 4.0 from 4 query_epoch 45 query: query(info 0'0 epoch_sent 45)) class_id 2immediate prio 255 cost 10 e45) queued 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 45 epoch_requested: 45 MQuery 4.0 from 4 query_epoch 45 query: query(info 0'0 epoch_sent 45)) class_id 2immediate prio 255 cost 10 e45) pg 0 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 45 handle_pg_query_nopg 4.0 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 45 pg 4.0 dne 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 45 get_con_osd_cluster to osd.4 from_epoch 45 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 45 get_nextmap_reserved map_reservations: {45=1} 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 45 release_map epoch: 45 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 45 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 41 -> 45 (shared) 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 45 send_incremental_map epoch range: (42, 45] to v2:127.0.0.1:6836/864727417 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 45 add_map_inc_bl 42 582 bytes 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 45 add_map_inc_bl 43 574 bytes 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 45 add_map_inc_bl 44 574 bytes 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 45 add_map_inc_bl 45 591 bytes 2026-02-11T23:30:24.482+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(42..45 src has 1..45) -- 0x46ea540 con 0x3e4a000 2026-02-11T23:30:24.482+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:45 sent:45 4.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e45/45) -- 0x3382300 con 0x3e4a000 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:24.482+0000 7fac5e983640 7 osd.0 45 activate_map version 45 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 45 epoch_requested: 45 NullEvt) class_id 2immediate prio 255 cost 10 e45) queued 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 45 epoch_requested: 45 NullEvt) class_id 2immediate prio 255 cost 10 e45) pg 0x40f8000 2026-02-11T23:30:24.482+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:24.482+0000 7fac55971640 20 osd.0 45 agent_entry empty queue 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 42 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 41 -> 42 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 41 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 42 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 42 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 45 advance_pg new pool opts old pool opts 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 42 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 43 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 42 -> 43 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 42 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 45 advance_pg new pool opts old pool opts 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 44 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 43 -> 44 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 43 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 45 advance_pg new pool opts old pool opts 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 45 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 44 -> 45 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 44 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 45 advance_pg new pool opts old pool opts 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 42..45 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 45 2026-02-11T23:30:24.482+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:24.482+0000 7fac55170640 10 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 45 epoch_requested: 45 NullEvt 2026-02-11T23:30:24.482+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:24.482+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:24.582+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 5 ==== PGlog(4.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.531513214s int 16.000000000s) e46/45) ==== 1172+0+0 (unknown 0 0 0) 0x3901800 con 0x3e4a000 2026-02-11T23:30:24.582+0000 7fac703fa640 15 osd.0 45 enqueue_peering_evt 4.0 epoch_sent: 46 epoch_requested: 45 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.531513214s int 16.000000000s) +create_info 2026-02-11T23:30:24.582+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 45 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.531513214s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e46) 2026-02-11T23:30:24.582+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:24.582+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 45 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.531513214s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e46) queued 2026-02-11T23:30:24.582+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:24.582+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 45 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.531513214s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e46) pg 0 2026-02-11T23:30:24.582+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 map 46 > 45 2026-02-11T23:30:24.582+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 46, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 45 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.531513214s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e46) 2026-02-11T23:30:24.582+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:24.582+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.026+0000 7fac68beb640 10 osd.0 45 tick 2026-02-11T23:30:25.026+0000 7fac68beb640 20 osd.0 45 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:25.054+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 4 ==== osd_ping(ping e46 up_from 21 ping_stamp 2026-02-11T23:30:25.055746+0000/8.003252983s send_stamp 8.003252983s delta_ub -9.886757851s) ==== 2033+0+0 (crc 0 0 0) 0x3b1aa80 con 0x3f57400 2026-02-11T23:30:25.054+0000 7fac703fa640 20 osd.0 45 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886992455s,-9.886757851s]) 2026-02-11T23:30:25.054+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e45 up_from 5 ping_stamp 2026-02-11T23:30:25.055746+0000/8.003252983s send_stamp 17.890245438s delta_ub 9.886992455s) -- 0x3b1b500 con 0x3f57400 2026-02-11T23:30:25.054+0000 7fac703fa640 20 osd.0 45 get_con_osd_cluster to osd.4 from_epoch 45 2026-02-11T23:30:25.054+0000 7fac703fa640 20 osd.0 45 get_nextmap_reserved map_reservations: {45=1} 2026-02-11T23:30:25.054+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 4 ==== osd_ping(ping e46 up_from 21 ping_stamp 2026-02-11T23:30:25.055746+0000/8.003252983s send_stamp 8.003252983s delta_ub -9.886757851s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b980 con 0x3f57800 2026-02-11T23:30:25.054+0000 7fac703fa640 20 osd.0 45 release_map epoch: 45 2026-02-11T23:30:25.054+0000 7fac703fa640 10 osd.0 45 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 45 to ping map epoch of 46 2026-02-11T23:30:25.054+0000 7fac703fa640 10 osd.0 45 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 45 is not newer than session's projected_epoch of 46 2026-02-11T23:30:25.054+0000 7fac71bfd640 20 osd.0 45 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887017250s,-9.886757851s]) 2026-02-11T23:30:25.054+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e45 up_from 5 ping_stamp 2026-02-11T23:30:25.055746+0000/8.003252983s send_stamp 17.890268326s delta_ub 9.887017250s) -- 0x3b1a600 con 0x3f57800 2026-02-11T23:30:25.054+0000 7fac71bfd640 20 osd.0 45 get_con_osd_cluster to osd.4 from_epoch 45 2026-02-11T23:30:25.054+0000 7fac71bfd640 20 osd.0 45 get_nextmap_reserved map_reservations: {45=1} 2026-02-11T23:30:25.054+0000 7fac71bfd640 20 osd.0 45 release_map epoch: 45 2026-02-11T23:30:25.054+0000 7fac71bfd640 10 osd.0 45 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 45 is not newer than session's projected_epoch of 46 2026-02-11T23:30:25.054+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 6 ==== osd_map(46..46 src has 1..46) ==== 296+0+0 (unknown 0 0 0) 0x46ea540 con 0x3e4a000 2026-02-11T23:30:25.054+0000 7fac606e4640 20 osd.0 45 OSD::ms_dispatch: osd_map(46..46 src has 1..46) 2026-02-11T23:30:25.054+0000 7fac606e4640 20 osd.0 45 _dispatch 0x46ea540 osd_map(46..46 src has 1..46) 2026-02-11T23:30:25.054+0000 7fac606e4640 3 osd.0 45 handle_osd_map epochs [46,46], i have 45, src has [1,46] 2026-02-11T23:30:25.054+0000 7fac606e4640 10 osd.0 45 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:25.054+0000 7fac606e4640 10 osd.0 45 handle_osd_map got inc map for epoch 46 2026-02-11T23:30:25.054+0000 7fac606e4640 20 osd.0 45 got_full_map 46, nothing requested 2026-02-11T23:30:25.054+0000 7fac606e4640 20 osd.0 45 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:25.054+0000 7fac606e4640 20 osd.0 45 handle_osd_map pg_num_history pg_num_history(e46 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1}} deleted_pools ) 2026-02-11T23:30:25.054+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {46={}} 2026-02-11T23:30:25.054+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:25.054+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e46 maps [1~46] lci=[5,46] tlb=1) 2026-02-11T23:30:25.055+0000 7fac5e983640 10 osd.0 45 _committed_osd_maps 46..46 2026-02-11T23:30:25.055+0000 7fac5e983640 10 osd.0 45 advance to epoch 46 (<= last 46 <= newest_map 46) 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0 46 consume_map version 46 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0 45 await_reserved_maps epoch:46 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0 45 await_reserved_maps done 0 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0 46 consume_map 46 -- publish done 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 4.0 has no pg and waiting_for_split 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0 46 identify_splits_and_merges 1.0 e45 to e46 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0 46 identify_splits_and_merges 1.0 e45 to e46 pg_nums {24=1} 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0 46 prune_pg_created 2026-02-11T23:30:25.055+0000 7fac5e983640 10 osd.0:0.consume_map 46 (was 45) 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 46 <= 46, requeueing 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {46=} 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:25.055+0000 7fac5e983640 10 osd.0:1.consume_map 46 (was 45) 2026-02-11T23:30:25.055+0000 7fac5e983640 10 osd.0:2.consume_map 46 (was 45) 2026-02-11T23:30:25.055+0000 7fac5e983640 10 osd.0:3.consume_map 46 (was 45) 2026-02-11T23:30:25.055+0000 7fac5e983640 10 osd.0:4.consume_map 46 (was 45) 2026-02-11T23:30:25.055+0000 7fac5e983640 10 osd.0:5.consume_map 46 (was 45) 2026-02-11T23:30:25.055+0000 7fac5e983640 10 osd.0:6.consume_map 46 (was 45) 2026-02-11T23:30:25.055+0000 7fac5e983640 10 osd.0:7.consume_map 46 (was 45) 2026-02-11T23:30:25.055+0000 7fac5e983640 10 osd.0 46 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 45 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.531513214s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e46) queued 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.055+0000 7fac5e983640 15 osd.0 46 enqueue_peering_evt 1.0 epoch_sent: 46 epoch_requested: 46 NullEvt 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 45 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.531513214s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e46) pg 0 2026-02-11T23:30:25.055+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 NullEvt) class_id 2 prio 255 cost 10 e46) 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 no pg, should create on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 45 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.531513214s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e46) 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 46 _make_pg 4.0 2026-02-11T23:30:25.055+0000 7fac51168640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:30:25.055+0000 7fac51168640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:30:25.055+0000 7fac51168640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {"0000000000000004."}] update_bits prefix updated 2026-02-11T23:30:25.055+0000 7fac51168640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {"0000000000000004."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:30:25.055+0000 7fac51168640 5 osd.0 pg_epoch: 45 pg[4.0(unlocked)] enter Initial 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 pg_epoch: 45 pg[4.0(unlocked)] enter NotTrimming 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role 1 up [4,0,3] acting [4,0,3] history ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45 past_intervals ([0,0] all_participants={} intervals=[]) 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4,0,3]+[4,0,3] 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x3863810] 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 17.891292572s + 0.000000000s) 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:25.055+0000 7fac51168640 15 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:25.055+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:25.055+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:25.055+0000 7fac51168640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:30:25.055+0000 7fac51168640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:30:25.055+0000 7fac51168640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:30:25.055+0000 7fac51168640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:30:25.055+0000 7fac51168640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:30:25.055+0000 7fac51168640 10 end of _write_log_and_missing 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:30:25.055+0000 7fac51168640 5 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] exit Initial 0.000093 0 0.000000 2026-02-11T23:30:25.055+0000 7fac51168640 5 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 45 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 45..45 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:25.055+0000 7fac51168640 5 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] exit Reset 0.000012 1 0.000012 2026-02-11T23:30:25.055+0000 7fac51168640 5 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:30:25.055+0000 7fac51168640 5 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:30:25.055+0000 7fac51168640 1 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:30:25.055+0000 7fac51168640 5 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:25.055+0000 7fac51168640 5 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 45 while current is 45 2026-02-11T23:30:25.055+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 46 handle_pg_create_info new pg pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0:0._attach_pg 4.0 0x4856000 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 46 identify_splits_and_merges 4.0 e45 to e46 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 46 identify_splits_and_merges 4.0 e45 to e46 pg_nums {45=1} 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0:0._prime_splits 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] handle_advance_map: 46 2026-02-11T23:30:25.055+0000 7fac5e983640 7 osd.0 46 activate_map version 46 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 45 -> 46 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 45 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [4,0,3]/[4,0,3] -- 4/4 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 NullEvt) class_id 2immediate prio 255 cost 10 e46) queued 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 NullEvt) class_id 2immediate prio 255 cost 10 e46) pg 0x40f8000 2026-02-11T23:30:25.055+0000 7fac55971640 20 osd.0 46 agent_entry empty queue 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 46 advance_pg new pool opts old pool opts 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 46..46 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 45 while current is 46 2026-02-11T23:30:25.055+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 46 epoch_requested: 45 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 23.531513214s int 16.000000000s) +create_info 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] state: got info+log from osd.4 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:25.055+0000 7fac51168640 10 merge_log log((0'0,0'0], crt=0'0) from osd.4 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:25.055+0000 7fac51168640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:25.055+0000 7fac51168640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:25.055+0000 7fac51168640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 46 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 23.531513214s int 16.000000000s) 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 45 -> 46 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.887017250s -> ruub 33.418529510s 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] PeeringState::proc_lease readable_until_ub now 33.418529510s 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 pg_epoch: 45 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:25.055+0000 7fac51168640 5 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] exit Started/Stray 0.000077 3 0.000011 2026-02-11T23:30:25.055+0000 7fac51168640 5 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:25.055+0000 7fac51168640 5 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:25.055+0000 7fac51168640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 46 advance_pg new pool opts old pool opts 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:25.055+0000 7fac51168640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:25.055+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 46..46 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 46 2026-02-11T23:30:25.055+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.055+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 46 epoch_requested: 46 NullEvt 2026-02-11T23:30:25.055+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.055+0000 7fac55170640 15 osd.0 46 enqueue_peering_evt 4.0 epoch_sent: 46 epoch_requested: 46 ActivateCommitted from 46 processed at 46 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 ActivateCommitted from 46 processed at 46) class_id 2 prio 255 cost 10 e46) 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 ActivateCommitted from 46 processed at 46) class_id 2immediate prio 255 cost 10 e46) queued 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 ActivateCommitted from 46 processed at 46) class_id 2immediate prio 255 cost 10 e46) pg 0x4856000 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 46 epoch_requested: 46 ActivateCommitted from 46 processed at 46 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 unknown mbc={}] state: react 46 telling primary 2026-02-11T23:30:25.055+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 46 get_con_osd_cluster to osd.4 from_epoch 46 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 46 get_nextmap_reserved map_reservations: {46=1} 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 46 release_map epoch: 46 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 46 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 46 is not newer than session's projected_epoch of 46 2026-02-11T23:30:25.055+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_info2(4.0 4.0 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/0 les/c/f=46/0/0 sis=45) e46/46) -- 0x3382a00 con 0x3e4a000 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.055+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 7 ==== pg_lease(4.0 pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s) e46/46) ==== 58+0+0 (unknown 0 0 0) 0x3b1aa80 con 0x3e4a000 2026-02-11T23:30:25.055+0000 7fac703fa640 15 osd.0 46 enqueue_peering_evt 4.0 epoch_sent: 46 epoch_requested: 46 MLease epoch 46 from osd.4 pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s) 2026-02-11T23:30:25.055+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 MLease epoch 46 from osd.4 pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s)) class_id 2 prio 255 cost 10 e46) 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 MLease epoch 46 from osd.4 pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e46) queued 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 MLease epoch 46 from osd.4 pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e46) pg 0x4856000 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 46 epoch_requested: 46 MLease epoch 46 from osd.4 pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s) 2026-02-11T23:30:25.055+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.055+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s) 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.886757851s -> ru 33.418270111s 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 33.418270111s 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.887017250s -> ruub 33.891853333s 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 33.891853333s 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 46 get_con_osd_cluster to osd.4 from_epoch 46 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 46 get_nextmap_reserved map_reservations: {46=1} 2026-02-11T23:30:25.055+0000 7fac55170640 20 osd.0 46 release_map epoch: 46 2026-02-11T23:30:25.055+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_lease_ack(4.0 pg_lease_ack(ruub 24.004835129s) e46/46) -- 0x25d8dc0 con 0x3e4a000 2026-02-11T23:30:25.056+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.056+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 8 ==== pg_info2(4.0 4.0 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/0 les/c/f=46/0/0 sis=45) e46/46) ==== 1126+0+0 (unknown 0 0 0) 0x3382a00 con 0x3e4a000 2026-02-11T23:30:25.056+0000 7fac703fa640 15 osd.0 46 enqueue_peering_evt 4.0 epoch_sent: 46 epoch_requested: 46 MInfoRec from 4 info: 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/0 les/c/f=46/0/0 sis=45) pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s) 2026-02-11T23:30:25.056+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 MInfoRec from 4 info: 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/0 les/c/f=46/0/0 sis=45) pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s)) class_id 2 prio 255 cost 10 e46) 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 MInfoRec from 4 info: 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/0 les/c/f=46/0/0 sis=45) pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e46) queued 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 MInfoRec from 4 info: 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/0 les/c/f=46/0/0 sis=45) pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e46) pg 0x4856000 2026-02-11T23:30:25.056+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=0/0 les/c/f=0/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 46 epoch_requested: 46 MInfoRec from 4 info: 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/0 les/c/f=46/0/0 sis=45) pg_lease(ru 23.531513214s ub 24.004835129s int 16.000000000s) 2026-02-11T23:30:25.056+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/0 les/c/f=46/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=45/45 lis/c=45/0 les/c/f=46/0/0 sis=45 2026-02-11T23:30:25.056+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.056+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.056+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 9 ==== pg_info2(4.0 4.0 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) e46/46) ==== 1126+0+0 (unknown 0 0 0) 0x3382a00 con 0x3e4a000 2026-02-11T23:30:25.056+0000 7fac703fa640 15 osd.0 46 enqueue_peering_evt 4.0 epoch_sent: 46 epoch_requested: 46 MInfoRec from 4 info: 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) pg_lease(ru 24.004835129s ub 24.004835129s int 16.000000000s) 2026-02-11T23:30:25.056+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 MInfoRec from 4 info: 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) pg_lease(ru 24.004835129s ub 24.004835129s int 16.000000000s)) class_id 2 prio 255 cost 10 e46) 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 MInfoRec from 4 info: 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) pg_lease(ru 24.004835129s ub 24.004835129s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e46) queued 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 46 epoch_requested: 46 MInfoRec from 4 info: 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) pg_lease(ru 24.004835129s ub 24.004835129s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e46) pg 0x4856000 2026-02-11T23:30:25.056+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.056+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/0 les/c/f=46/0/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 46 epoch_requested: 46 MInfoRec from 4 info: 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) pg_lease(ru 24.004835129s ub 24.004835129s int 16.000000000s) 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:25.056+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.056+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.056+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:25.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:25.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:55.168306+0000) 2026-02-11T23:30:25.249+0000 7fac683ea640 10 osd.0 46 tick_without_osd_lock 2026-02-11T23:30:25.249+0000 7fac683ea640 0 osd.0 46 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:25.249+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:25.249+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:25.249+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:25.249+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.152 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:25.249+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:25.250, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:25.249+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:25.249+0000 7fac683ea640 10 osd.0 46 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:25.249+0000 7fac683ea640 20 osd.0 46 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:25.249+0000 7fac683ea640 10 osd.0 46 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:25.249+0000 7fac683ea640 20 osd.0 46 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:25.448+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 5 ==== osd_ping(ping e47 up_from 9 ping_stamp 2026-02-11T23:30:25.449613+0000/16.303176880s send_stamp 16.303176880s delta_ub -1.980759859s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a600 con 0x396a400 2026-02-11T23:30:25.448+0000 7fac71bfd640 20 osd.0 46 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980932355s,-1.980759859s]) 2026-02-11T23:30:25.448+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e46 up_from 5 ping_stamp 2026-02-11T23:30:25.449613+0000/16.303176880s send_stamp 18.284109116s delta_ub 1.980932355s) -- 0x3b1b980 con 0x396a400 2026-02-11T23:30:25.448+0000 7fac71bfd640 20 osd.0 46 get_con_osd_cluster to osd.1 from_epoch 46 2026-02-11T23:30:25.448+0000 7fac71bfd640 20 osd.0 46 get_nextmap_reserved map_reservations: {46=1} 2026-02-11T23:30:25.448+0000 7fac71bfd640 20 osd.0 46 release_map epoch: 46 2026-02-11T23:30:25.448+0000 7fac71bfd640 10 osd.0 46 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 41 to ping map epoch of 47 2026-02-11T23:30:25.448+0000 7fac71bfd640 10 osd.0 46 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 46 is not newer than session's projected_epoch of 47 2026-02-11T23:30:25.448+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 5 ==== osd_ping(ping e47 up_from 9 ping_stamp 2026-02-11T23:30:25.449613+0000/16.303176880s send_stamp 16.303176880s delta_ub -1.980759859s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bb00 con 0x396a000 2026-02-11T23:30:25.448+0000 7fac723fe640 20 osd.0 46 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980949879s,-1.980759859s]) 2026-02-11T23:30:25.448+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e46 up_from 5 ping_stamp 2026-02-11T23:30:25.449613+0000/16.303176880s send_stamp 18.284128189s delta_ub 1.980949879s) -- 0x3b1be00 con 0x396a000 2026-02-11T23:30:25.448+0000 7fac723fe640 20 osd.0 46 get_con_osd_cluster to osd.1 from_epoch 46 2026-02-11T23:30:25.448+0000 7fac723fe640 20 osd.0 46 get_nextmap_reserved map_reservations: {46=1} 2026-02-11T23:30:25.448+0000 7fac723fe640 20 osd.0 46 release_map epoch: 46 2026-02-11T23:30:25.448+0000 7fac723fe640 10 osd.0 46 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 46 is not newer than session's projected_epoch of 47 2026-02-11T23:30:25.448+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 16 ==== osd_map(47..47 src has 1..47) ==== 618+0+0 (unknown 0 0 0) 0x3940c40 con 0x3752000 2026-02-11T23:30:25.448+0000 7fac606e4640 20 osd.0 46 OSD::ms_dispatch: osd_map(47..47 src has 1..47) 2026-02-11T23:30:25.448+0000 7fac606e4640 20 osd.0 46 _dispatch 0x3940c40 osd_map(47..47 src has 1..47) 2026-02-11T23:30:25.448+0000 7fac606e4640 3 osd.0 46 handle_osd_map epochs [47,47], i have 46, src has [1,47] 2026-02-11T23:30:25.448+0000 7fac606e4640 10 osd.0 46 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:25.448+0000 7fac606e4640 10 osd.0 46 handle_osd_map got inc map for epoch 47 2026-02-11T23:30:25.448+0000 7fac606e4640 20 osd.0 46 got_full_map 47, nothing requested 2026-02-11T23:30:25.448+0000 7fac606e4640 20 osd.0 46 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:25.448+0000 7fac606e4640 20 osd.0 46 handle_osd_map pg_num_history pg_num_history(e47 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1}} deleted_pools ) 2026-02-11T23:30:25.448+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {47={}} 2026-02-11T23:30:25.448+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:25.448+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e47 maps [1~47] lci=[5,47] tlb=1) 2026-02-11T23:30:25.448+0000 7fac5e983640 10 osd.0 46 _committed_osd_maps 47..47 2026-02-11T23:30:25.448+0000 7fac5e983640 10 osd.0 46 advance to epoch 47 (<= last 47 <= newest_map 47) 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0 47 consume_map version 47 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0 46 await_reserved_maps epoch:47 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0 46 await_reserved_maps done 0 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0 47 consume_map 47 -- publish done 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0 47 identify_splits_and_merges 4.0 e46 to e47 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0 47 identify_splits_and_merges 4.0 e46 to e47 pg_nums {45=1} 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0 47 identify_splits_and_merges 1.0 e46 to e47 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0 47 identify_splits_and_merges 1.0 e46 to e47 pg_nums {24=1} 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0 47 prune_pg_created 2026-02-11T23:30:25.448+0000 7fac5e983640 10 osd.0:0.consume_map 47 (was 46) 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:25.448+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:25.448+0000 7fac5e983640 10 osd.0:1.consume_map 47 (was 46) 2026-02-11T23:30:25.448+0000 7fac5e983640 10 osd.0:2.consume_map 47 (was 46) 2026-02-11T23:30:25.448+0000 7fac5e983640 10 osd.0:3.consume_map 47 (was 46) 2026-02-11T23:30:25.448+0000 7fac5e983640 10 osd.0:4.consume_map 47 (was 46) 2026-02-11T23:30:25.449+0000 7fac5e983640 10 osd.0:5.consume_map 47 (was 46) 2026-02-11T23:30:25.449+0000 7fac5e983640 10 osd.0:6.consume_map 47 (was 46) 2026-02-11T23:30:25.449+0000 7fac5e983640 10 osd.0:7.consume_map 47 (was 46) 2026-02-11T23:30:25.449+0000 7fac5e983640 10 osd.0 47 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:25.449+0000 7fac5e983640 15 osd.0 47 enqueue_peering_evt 4.0 epoch_sent: 47 epoch_requested: 47 NullEvt 2026-02-11T23:30:25.449+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 47 epoch_requested: 47 NullEvt) class_id 2 prio 255 cost 10 e47) 2026-02-11T23:30:25.449+0000 7fac5e983640 15 osd.0 47 enqueue_peering_evt 1.0 epoch_sent: 47 epoch_requested: 47 NullEvt 2026-02-11T23:30:25.449+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 47 epoch_requested: 47 NullEvt) class_id 2 prio 255 cost 10 e47) 2026-02-11T23:30:25.449+0000 7fac5e983640 7 osd.0 47 activate_map version 47 2026-02-11T23:30:25.449+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.449+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 47 epoch_requested: 47 NullEvt) class_id 2immediate prio 255 cost 10 e47) queued 2026-02-11T23:30:25.449+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.449+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 47 epoch_requested: 47 NullEvt) class_id 2immediate prio 255 cost 10 e47) pg 0x4856000 2026-02-11T23:30:25.449+0000 7fac55971640 20 osd.0 47 agent_entry empty queue 2026-02-11T23:30:25.449+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] handle_advance_map: 47 2026-02-11T23:30:25.449+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 46 -> 47 2026-02-11T23:30:25.449+0000 7fac51168640 10 osd.0 pg_epoch: 46 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0,3]/[4,0,3] -- 4/4 2026-02-11T23:30:25.449+0000 7fac51168640 10 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:25.449+0000 7fac51168640 10 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:25.449+0000 7fac51168640 10 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:25.449+0000 7fac51168640 20 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:25.449+0000 7fac51168640 20 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:25.449+0000 7fac51168640 20 osd.0 47 advance_pg new pool opts old pool opts 2026-02-11T23:30:25.449+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.449+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 47 epoch_requested: 47 NullEvt) class_id 2immediate prio 255 cost 10 e47) queued 2026-02-11T23:30:25.449+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.449+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 47 epoch_requested: 47 NullEvt) class_id 2immediate prio 255 cost 10 e47) pg 0x40f8000 2026-02-11T23:30:25.449+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 47 2026-02-11T23:30:25.449+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 46 -> 47 2026-02-11T23:30:25.449+0000 7fac55170640 10 osd.0 pg_epoch: 46 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:25.449+0000 7fac55170640 10 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:25.449+0000 7fac55170640 10 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:25.449+0000 7fac55170640 20 osd.0 47 advance_pg new pool opts old pool opts 2026-02-11T23:30:25.449+0000 7fac51168640 10 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] handle_activate_map: epoch range: 47..47 2026-02-11T23:30:25.449+0000 7fac55170640 10 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 47..47 2026-02-11T23:30:25.449+0000 7fac55170640 10 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:25.449+0000 7fac51168640 10 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:25.449+0000 7fac55170640 20 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 47 2026-02-11T23:30:25.449+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.449+0000 7fac51168640 20 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 46 while current is 47 2026-02-11T23:30:25.449+0000 7fac55170640 10 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 47 epoch_requested: 47 NullEvt 2026-02-11T23:30:25.449+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:25.449+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.449+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.449+0000 7fac51168640 10 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:25.449+0000 7fac51168640 10 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 47 epoch_requested: 47 NullEvt 2026-02-11T23:30:25.449+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:25.449+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.578+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 3 ==== osd_ping(ping e47 up_from 17 ping_stamp 2026-02-11T23:30:25.579372+0000/11.103408813s send_stamp 11.103408813s delta_ub -7.310237408s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b980 con 0x3f56800 2026-02-11T23:30:25.578+0000 7fac71bfd640 20 osd.0 47 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310453892s,-7.310237408s]) 2026-02-11T23:30:25.578+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e47 up_from 5 ping_stamp 2026-02-11T23:30:25.579372+0000/11.103408813s send_stamp 18.413862228s delta_ub 7.310453892s) -- 0x3b1a600 con 0x3f56800 2026-02-11T23:30:25.578+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 3 ==== osd_ping(ping e47 up_from 17 ping_stamp 2026-02-11T23:30:25.579372+0000/11.103408813s send_stamp 11.103408813s delta_ub -7.310237408s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x3f56c00 2026-02-11T23:30:25.578+0000 7fac71bfd640 20 osd.0 47 get_con_osd_cluster to osd.3 from_epoch 47 2026-02-11T23:30:25.578+0000 7fac71bfd640 20 osd.0 47 get_nextmap_reserved map_reservations: {47=1} 2026-02-11T23:30:25.578+0000 7fac71bfd640 20 osd.0 47 release_map epoch: 47 2026-02-11T23:30:25.578+0000 7fac71bfd640 10 osd.0 47 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 41 to ping map epoch of 47 2026-02-11T23:30:25.578+0000 7fac71bfd640 10 osd.0 47 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 47 is not newer than session's projected_epoch of 47 2026-02-11T23:30:25.578+0000 7fac723fe640 20 osd.0 47 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310479641s,-7.310237408s]) 2026-02-11T23:30:25.578+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e47 up_from 5 ping_stamp 2026-02-11T23:30:25.579372+0000/11.103408813s send_stamp 18.413888931s delta_ub 7.310479641s) -- 0x3b1bb00 con 0x3f56c00 2026-02-11T23:30:25.578+0000 7fac723fe640 20 osd.0 47 get_con_osd_cluster to osd.3 from_epoch 47 2026-02-11T23:30:25.578+0000 7fac723fe640 20 osd.0 47 get_nextmap_reserved map_reservations: {47=1} 2026-02-11T23:30:25.578+0000 7fac723fe640 20 osd.0 47 release_map epoch: 47 2026-02-11T23:30:25.578+0000 7fac723fe640 10 osd.0 47 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 47 is not newer than session's projected_epoch of 47 2026-02-11T23:30:25.768+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 11 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 48) e48/48) ==== 158+0+0 (unknown 0 0 0) 0x3f3f680 con 0x3e4a000 2026-02-11T23:30:25.768+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 10 ==== osd_map(47..48 src has 1..48) ==== 1200+0+0 (unknown 0 0 0) 0x3941500 con 0x3e4a000 2026-02-11T23:30:25.768+0000 7fac606e4640 20 osd.0 47 OSD::ms_dispatch: osd_map(47..48 src has 1..48) 2026-02-11T23:30:25.768+0000 7fac606e4640 20 osd.0 47 _dispatch 0x3941500 osd_map(47..48 src has 1..48) 2026-02-11T23:30:25.768+0000 7fac703fa640 15 osd.0 47 enqueue_peering_evt 4.0 epoch_sent: 48 epoch_requested: 48 MQuery 4.0 from 4 query_epoch 48 query: query(info 0'0 epoch_sent 48) 2026-02-11T23:30:25.768+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 MQuery 4.0 from 4 query_epoch 48 query: query(info 0'0 epoch_sent 48)) class_id 2 prio 255 cost 10 e48) 2026-02-11T23:30:25.768+0000 7fac606e4640 3 osd.0 47 handle_osd_map epochs [47,48], i have 47, src has [1,48] 2026-02-11T23:30:25.768+0000 7fac606e4640 10 osd.0 47 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:25.768+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.768+0000 7fac606e4640 10 osd.0 47 handle_osd_map got inc map for epoch 48 2026-02-11T23:30:25.768+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 MQuery 4.0 from 4 query_epoch 48 query: query(info 0'0 epoch_sent 48)) class_id 2immediate prio 255 cost 10 e48) queued 2026-02-11T23:30:25.768+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.768+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 MQuery 4.0 from 4 query_epoch 48 query: query(info 0'0 epoch_sent 48)) class_id 2immediate prio 255 cost 10 e48) pg 0x4856000 2026-02-11T23:30:25.768+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 48, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 MQuery 4.0 from 4 query_epoch 48 query: query(info 0'0 epoch_sent 48)) class_id 2immediate prio 255 cost 10 e48) 2026-02-11T23:30:25.768+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.768+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.768+0000 7fac606e4640 20 osd.0 47 got_full_map 48, nothing requested 2026-02-11T23:30:25.768+0000 7fac606e4640 20 osd.0 47 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:25.768+0000 7fac606e4640 20 osd.0 47 handle_osd_map pg_num_history pg_num_history(e48 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1}} deleted_pools ) 2026-02-11T23:30:25.768+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {48={}} 2026-02-11T23:30:25.768+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:25.768+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e48 maps [1~48] lci=[5,48] tlb=1) 2026-02-11T23:30:25.769+0000 7fac5e983640 10 osd.0 47 _committed_osd_maps 48..48 2026-02-11T23:30:25.769+0000 7fac5e983640 10 osd.0 47 advance to epoch 48 (<= last 48 <= newest_map 48) 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0 48 consume_map version 48 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0 47 await_reserved_maps epoch:48 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0 47 await_reserved_maps done 0 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0 48 consume_map 48 -- publish done 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0 48 identify_splits_and_merges 4.0 e47 to e48 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0 48 identify_splits_and_merges 4.0 e47 to e48 pg_nums {45=1} 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0 48 identify_splits_and_merges 1.0 e47 to e48 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0 48 identify_splits_and_merges 1.0 e47 to e48 pg_nums {24=1} 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0 48 prune_pg_created 2026-02-11T23:30:25.769+0000 7fac5e983640 10 osd.0:0.consume_map 48 (was 47) 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 48 <= 48, requeueing 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {48=} 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:25.769+0000 7fac5e983640 10 osd.0:1.consume_map 48 (was 47) 2026-02-11T23:30:25.769+0000 7fac5e983640 10 osd.0:2.consume_map 48 (was 47) 2026-02-11T23:30:25.769+0000 7fac5e983640 10 osd.0:3.consume_map 48 (was 47) 2026-02-11T23:30:25.769+0000 7fac5e983640 10 osd.0:4.consume_map 48 (was 47) 2026-02-11T23:30:25.769+0000 7fac5e983640 10 osd.0:5.consume_map 48 (was 47) 2026-02-11T23:30:25.769+0000 7fac5e983640 10 osd.0:6.consume_map 48 (was 47) 2026-02-11T23:30:25.769+0000 7fac5e983640 10 osd.0:7.consume_map 48 (was 47) 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 MQuery 4.0 from 4 query_epoch 48 query: query(info 0'0 epoch_sent 48)) class_id 2immediate prio 255 cost 10 e48) queued 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 MQuery 4.0 from 4 query_epoch 48 query: query(info 0'0 epoch_sent 48)) class_id 2immediate prio 255 cost 10 e48) pg 0x4856000 2026-02-11T23:30:25.769+0000 7fac5e983640 10 osd.0 48 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:25.769+0000 7fac5e983640 15 osd.0 48 enqueue_peering_evt 4.0 epoch_sent: 48 epoch_requested: 48 NullEvt 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 NullEvt) class_id 2 prio 255 cost 10 e48) 2026-02-11T23:30:25.769+0000 7fac5e983640 15 osd.0 48 enqueue_peering_evt 1.0 epoch_sent: 48 epoch_requested: 48 NullEvt 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] handle_advance_map: 48 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 47 -> 48 2026-02-11T23:30:25.769+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 NullEvt) class_id 2 prio 255 cost 10 e48) 2026-02-11T23:30:25.769+0000 7fac5e983640 7 osd.0 48 activate_map version 48 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 47 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:25.769+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.769+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 NullEvt) class_id 2immediate prio 255 cost 10 e48) queued 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4,0] newacting [4,0] 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:25.769+0000 7fac55971640 20 osd.0 48 agent_entry empty queue 2026-02-11T23:30:25.769+0000 7fac55170640 5 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.713742 8 0.000051 2026-02-11T23:30:25.769+0000 7fac55170640 5 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] exit Started/ReplicaActive 0.713746 0 0.000000 2026-02-11T23:30:25.769+0000 7fac55170640 5 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] exit Started 0.713836 0 0.000000 2026-02-11T23:30:25.769+0000 7fac55170640 5 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=45 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 48 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=48 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4,0] newacting [4,0] 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=48 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0,3] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 48 2026-02-11T23:30:25.769+0000 7fac55170640 15 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(45-47 up [4, 0, 3](4) acting [4, 0, 3](4)) up_thru 45 up_from 21 last_epoch_clean 46 interval(45-47 up [4, 0, 3](4) acting [4, 0, 3](4) maybe_went_rw) : primary up 21-45 includes interval 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=45) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([45,47] all_participants={0, 3, 4} intervals=[([45,47] acting={0, 3, 4})]) 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4,0]+[4,0] 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [0x3863810] 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 33.891853333s (mnow 18.605211258s + 15.286640167s) 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] cancel_recovery 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] clear_recovery_state 2026-02-11T23:30:25.769+0000 7fac55170640 15 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:25.769+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:25.769+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:25.769+0000 7fac55170640 1 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::start_peering_interval up [4,0,3] -> [4,0], acting [4,0,3] -> [4,0], acting_primary 4 -> 4, up_primary 4 -> 4, role 1 -> 1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] clear_primary_state 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] release_backoffs [4:00000000::::0,MAX) 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:25.769+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:25.769+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] agent_stop 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] on_change 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] on_change_cleanup 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] on_change 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] exit NotTrimming 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] enter NotTrimming 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] hit_set_clear 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] agent_stop 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 48 advance_pg new pool opts old pool opts 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] handle_activate_map: epoch range: 48..48 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286640167s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:25.769+0000 7fac55170640 5 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286576271s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] exit Reset 0.000122 1 0.000112 2026-02-11T23:30:25.769+0000 7fac55170640 5 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286576271s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] enter Started 2026-02-11T23:30:25.769+0000 7fac55170640 5 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286576271s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] enter Start 2026-02-11T23:30:25.769+0000 7fac55170640 1 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286576271s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:25.769+0000 7fac55170640 5 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286576271s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:25.769+0000 7fac55170640 5 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286576271s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] enter Started/Stray 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286576271s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 46 while current is 48 2026-02-11T23:30:25.769+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286576271s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286576271s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] do_peering_event: epoch_sent: 48 epoch_requested: 48 MQuery 4.0 from 4 query_epoch 48 query: query(info 0'0 epoch_sent 48) 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:25.769+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 48 get_con_osd_cluster to osd.4 from_epoch 48 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 48 get_nextmap_reserved map_reservations: {48=1} 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 48 release_map epoch: 48 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 48 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 46 -> 48 (shared) 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 48 send_incremental_map epoch range: (47, 48] to v2:127.0.0.1:6836/864727417 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 48 add_map_inc_bl 47 574 bytes 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 48 add_map_inc_bl 48 574 bytes 2026-02-11T23:30:25.769+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(47..48 src has 1..48) -- 0x3941dc0 con 0x3e4a000 2026-02-11T23:30:25.769+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:48 sent:48 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286576271s) ([45,47] all_participants={0, 3, 4} intervals=[([45,47] acting={0, 3, 4})])) e48/48) -- 0x4868000 con 0x3e4a000 2026-02-11T23:30:25.769+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:48 sent:48 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) ([45,47] all_participants={0, 3, 4} intervals=[([45,47] acting={0, 3, 4})])) e48/48) -- 0x4868700 con 0x3e4a000 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 NullEvt) class_id 2immediate prio 255 cost 10 e48) queued 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 NullEvt) class_id 2immediate prio 255 cost 10 e48) pg 0x40f8000 2026-02-11T23:30:25.769+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.769+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 48 epoch_requested: 48 NullEvt) class_id 2immediate prio 255 cost 10 e48) pg 0x4856000 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 48 2026-02-11T23:30:25.769+0000 7fac51168640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] do_peering_event: epoch_sent: 48 epoch_requested: 48 NullEvt 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 47 -> 48 2026-02-11T23:30:25.769+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 47 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:25.769+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 48 advance_pg new pool opts old pool opts 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 48..48 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 48 2026-02-11T23:30:25.769+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.769+0000 7fac55170640 10 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 48 epoch_requested: 48 NullEvt 2026-02-11T23:30:25.769+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.769+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.769+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.868+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 12 ==== PGlog(4.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.817348480s int 16.000000000s) e49/48) ==== 1164+0+0 (unknown 0 0 0) 0x3902800 con 0x3e4a000 2026-02-11T23:30:25.868+0000 7fac703fa640 15 osd.0 48 enqueue_peering_evt 4.0 epoch_sent: 49 epoch_requested: 48 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.817348480s int 16.000000000s) +create_info 2026-02-11T23:30:25.868+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 48 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.817348480s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e49) 2026-02-11T23:30:25.868+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:25.868+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 48 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.817348480s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e49) queued 2026-02-11T23:30:25.868+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:25.868+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 48 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.817348480s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e49) pg 0x4856000 2026-02-11T23:30:25.868+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 49, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 48 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.817348480s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e49) 2026-02-11T23:30:25.868+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:25.868+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.045+0000 7fac68beb640 10 osd.0 48 tick 2026-02-11T23:30:26.045+0000 7fac68beb640 20 osd.0 48 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:26.154+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 5 ==== osd_ping(ping e49 up_from 21 ping_stamp 2026-02-11T23:30:26.155942+0000/9.103448868s send_stamp 9.103448868s delta_ub -9.886684418s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a600 con 0x3f57800 2026-02-11T23:30:26.154+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 5 ==== osd_ping(ping e49 up_from 21 ping_stamp 2026-02-11T23:30:26.155942+0000/9.103448868s send_stamp 9.103448868s delta_ub -9.886684418s) ==== 2033+0+0 (crc 0 0 0) 0x3b1aa80 con 0x3f57400 2026-02-11T23:30:26.154+0000 7fac71bfd640 20 osd.0 48 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887002945s,-9.886684418s]) 2026-02-11T23:30:26.154+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e48 up_from 5 ping_stamp 2026-02-11T23:30:26.155942+0000/9.103448868s send_stamp 18.990449905s delta_ub 9.887002945s) -- 0x3b1b980 con 0x3f57800 2026-02-11T23:30:26.154+0000 7fac71bfd640 20 osd.0 48 get_con_osd_cluster to osd.4 from_epoch 48 2026-02-11T23:30:26.154+0000 7fac71bfd640 20 osd.0 48 get_nextmap_reserved map_reservations: {48=1} 2026-02-11T23:30:26.154+0000 7fac71bfd640 20 osd.0 48 release_map epoch: 48 2026-02-11T23:30:26.154+0000 7fac71bfd640 10 osd.0 48 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 48 to ping map epoch of 49 2026-02-11T23:30:26.154+0000 7fac71bfd640 10 osd.0 48 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 48 is not newer than session's projected_epoch of 49 2026-02-11T23:30:26.154+0000 7fac703fa640 20 osd.0 48 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887022972s,-9.886684418s]) 2026-02-11T23:30:26.154+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e48 up_from 5 ping_stamp 2026-02-11T23:30:26.155942+0000/9.103448868s send_stamp 18.990472794s delta_ub 9.887022972s) -- 0x3b1a300 con 0x3f57400 2026-02-11T23:30:26.154+0000 7fac703fa640 20 osd.0 48 get_con_osd_cluster to osd.4 from_epoch 48 2026-02-11T23:30:26.154+0000 7fac703fa640 20 osd.0 48 get_nextmap_reserved map_reservations: {48=1} 2026-02-11T23:30:26.154+0000 7fac703fa640 20 osd.0 48 release_map epoch: 48 2026-02-11T23:30:26.154+0000 7fac703fa640 10 osd.0 48 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 48 is not newer than session's projected_epoch of 49 2026-02-11T23:30:26.154+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 13 ==== osd_map(49..49 src has 1..49) ==== 626+0+0 (unknown 0 0 0) 0x3941dc0 con 0x3e4a000 2026-02-11T23:30:26.154+0000 7fac606e4640 20 osd.0 48 OSD::ms_dispatch: osd_map(49..49 src has 1..49) 2026-02-11T23:30:26.154+0000 7fac606e4640 20 osd.0 48 _dispatch 0x3941dc0 osd_map(49..49 src has 1..49) 2026-02-11T23:30:26.154+0000 7fac606e4640 3 osd.0 48 handle_osd_map epochs [49,49], i have 48, src has [1,49] 2026-02-11T23:30:26.154+0000 7fac606e4640 10 osd.0 48 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:26.154+0000 7fac606e4640 10 osd.0 48 handle_osd_map got inc map for epoch 49 2026-02-11T23:30:26.155+0000 7fac606e4640 20 osd.0 48 got_full_map 49, nothing requested 2026-02-11T23:30:26.155+0000 7fac606e4640 20 osd.0 48 trim_maps: min=0 oldest_map=1 2026-02-11T23:30:26.155+0000 7fac606e4640 20 osd.0 48 handle_osd_map pg_num_history pg_num_history(e49 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1}} deleted_pools ) 2026-02-11T23:30:26.155+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {49={}} 2026-02-11T23:30:26.155+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:26.155+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e49 maps [1~49] lci=[5,49] tlb=1) 2026-02-11T23:30:26.155+0000 7fac5e983640 10 osd.0 48 _committed_osd_maps 49..49 2026-02-11T23:30:26.155+0000 7fac5e983640 10 osd.0 48 advance to epoch 49 (<= last 49 <= newest_map 49) 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0 49 consume_map version 49 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0 48 await_reserved_maps epoch:49 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0 48 await_reserved_maps done 0 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0 49 consume_map 49 -- publish done 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0 49 identify_splits_and_merges 4.0 e48 to e49 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0 49 identify_splits_and_merges 4.0 e48 to e49 pg_nums {45=1} 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0 49 identify_splits_and_merges 1.0 e48 to e49 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0 49 identify_splits_and_merges 1.0 e48 to e49 pg_nums {24=1} 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0 49 prune_pg_created 2026-02-11T23:30:26.155+0000 7fac5e983640 10 osd.0:0.consume_map 49 (was 48) 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 49 <= 49, requeueing 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {49=} 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:26.155+0000 7fac5e983640 10 osd.0:1.consume_map 49 (was 48) 2026-02-11T23:30:26.155+0000 7fac5e983640 10 osd.0:2.consume_map 49 (was 48) 2026-02-11T23:30:26.155+0000 7fac5e983640 10 osd.0:3.consume_map 49 (was 48) 2026-02-11T23:30:26.155+0000 7fac5e983640 10 osd.0:4.consume_map 49 (was 48) 2026-02-11T23:30:26.155+0000 7fac5e983640 10 osd.0:5.consume_map 49 (was 48) 2026-02-11T23:30:26.155+0000 7fac5e983640 10 osd.0:6.consume_map 49 (was 48) 2026-02-11T23:30:26.155+0000 7fac5e983640 10 osd.0:7.consume_map 49 (was 48) 2026-02-11T23:30:26.155+0000 7fac5e983640 10 osd.0 49 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.155+0000 7fac5e983640 15 osd.0 49 enqueue_peering_evt 4.0 epoch_sent: 49 epoch_requested: 49 NullEvt 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 48 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.817348480s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e49) queued 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 NullEvt) class_id 2 prio 255 cost 10 e49) 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 48 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.817348480s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e49) pg 0x4856000 2026-02-11T23:30:26.155+0000 7fac5e983640 15 osd.0 49 enqueue_peering_evt 1.0 epoch_sent: 49 epoch_requested: 49 NullEvt 2026-02-11T23:30:26.155+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 NullEvt) class_id 2 prio 255 cost 10 e49) 2026-02-11T23:30:26.155+0000 7fac5e983640 7 osd.0 49 activate_map version 49 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] handle_advance_map: 49 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 NullEvt) class_id 2immediate prio 255 cost 10 e49) queued 2026-02-11T23:30:26.155+0000 7fac55971640 20 osd.0 49 agent_entry empty queue 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 48 -> 49 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 48 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] state: Started advmap 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] hit_set_clear 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] agent_stop 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 49 advance_pg new pool opts old pool opts 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] handle_activate_map: epoch range: 49..49 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=15.286521912s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 48 while current is 49 2026-02-11T23:30:26.155+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] do_peering_event: epoch_sent: 49 epoch_requested: 48 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 24.817348480s int 16.000000000s) +create_info 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] state: got info+log from osd.4 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:26.155+0000 7fac51168640 10 merge_log log((0'0,0'0], crt=0'0) from osd.4 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:26.155+0000 7fac51168640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:26.155+0000 7fac51168640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:26.155+0000 7fac51168640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 24.817348480s int 16.000000000s) 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.887022972s -> ruub 34.704372406s 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] PeeringState::proc_lease readable_until_ub now 34.704372406s 2026-02-11T23:30:26.155+0000 7fac51168640 5 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] exit Started/Stray 0.386167 5 0.000045 2026-02-11T23:30:26.155+0000 7fac51168640 5 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:26.155+0000 7fac51168640 5 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown NOTIFY pruub 33.891853333s@ mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:26.155+0000 7fac51168640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:26.155+0000 7fac51168640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] state: Activate Finished 2026-02-11T23:30:26.155+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 49 get_con_osd_cluster to osd.4 from_epoch 49 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 49 get_nextmap_reserved map_reservations: {49=1} 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 49 release_map epoch: 49 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 49 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 49 is not newer than session's projected_epoch of 49 2026-02-11T23:30:26.155+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:49 sent:49 4.0( empty local-lis/les=45/46 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) ([45,47] all_participants={0, 3, 4} intervals=[([45,47] acting={0, 3, 4})])) e49/49) -- 0x4868000 con 0x3e4a000 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 NullEvt) class_id 2immediate prio 255 cost 10 e49) pg 0x4856000 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 NullEvt) class_id 2immediate prio 255 cost 10 e49) queued 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.155+0000 7fac55170640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] do_peering_event: epoch_sent: 49 epoch_requested: 49 NullEvt 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 NullEvt) class_id 2immediate prio 255 cost 10 e49) pg 0x40f8000 2026-02-11T23:30:26.155+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 49 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 48 -> 49 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 48 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 49 advance_pg new pool opts old pool opts 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 49..49 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 49 2026-02-11T23:30:26.155+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.155+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 49 epoch_requested: 49 NullEvt 2026-02-11T23:30:26.155+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.155+0000 7fac55170640 15 osd.0 49 enqueue_peering_evt 4.0 epoch_sent: 49 epoch_requested: 49 ActivateCommitted from 49 processed at 49 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 ActivateCommitted from 49 processed at 49) class_id 2 prio 255 cost 10 e49) 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 ActivateCommitted from 49 processed at 49) class_id 2immediate prio 255 cost 10 e49) queued 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 49 epoch_requested: 49 ActivateCommitted from 49 processed at 49) class_id 2immediate prio 255 cost 10 e49) pg 0x4856000 2026-02-11T23:30:26.155+0000 7fac55170640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] do_peering_event: epoch_sent: 49 epoch_requested: 49 ActivateCommitted from 49 processed at 49 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.155+0000 7fac55170640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 unknown pruub 33.891853333s@ mbc={}] state: react 49 telling primary 2026-02-11T23:30:26.155+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 49 get_con_osd_cluster to osd.4 from_epoch 49 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 49 get_nextmap_reserved map_reservations: {49=1} 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 49 release_map epoch: 49 2026-02-11T23:30:26.155+0000 7fac55170640 10 osd.0 49 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 49 is not newer than session's projected_epoch of 49 2026-02-11T23:30:26.155+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_info2(4.0 4.0 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48 pruub=14.900403976s) e49/49) -- 0x4868e00 con 0x3e4a000 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.155+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 14 ==== pg_lease(4.0 pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s) e50/50) ==== 58+0+0 (unknown 0 0 0) 0x3b1a300 con 0x3e4a000 2026-02-11T23:30:26.155+0000 7fac703fa640 15 osd.0 49 enqueue_peering_evt 4.0 epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s) 2026-02-11T23:30:26.155+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2 prio 255 cost 10 e50) 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) pg 0x4856000 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 50, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.155+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 15 ==== pg_info2(4.0 4.0 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48) e50/50) ==== 1118+0+0 (unknown 0 0 0) 0x4868e00 con 0x3e4a000 2026-02-11T23:30:26.155+0000 7fac703fa640 15 osd.0 49 enqueue_peering_evt 4.0 epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s) 2026-02-11T23:30:26.155+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2 prio 255 cost 10 e50) 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {50=} 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {50=} 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) pg 0x4856000 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 50, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) 2026-02-11T23:30:26.155+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.155+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.156+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 16 ==== pg_info2(4.0 4.0 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) e50/50) ==== 1118+0+0 (unknown 0 0 0) 0x4868e00 con 0x3e4a000 2026-02-11T23:30:26.156+0000 7fac703fa640 15 osd.0 49 enqueue_peering_evt 4.0 epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s) 2026-02-11T23:30:26.156+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2 prio 255 cost 10 e50) 2026-02-11T23:30:26.156+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {50=} 2026-02-11T23:30:26.156+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:26.156+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {50=} 2026-02-11T23:30:26.156+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) pg 0x4856000 2026-02-11T23:30:26.156+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 50, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) 2026-02-11T23:30:26.156+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.156+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:26.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:26.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:56.168376+0000) 2026-02-11T23:30:26.260+0000 7fac683ea640 10 osd.0 49 tick_without_osd_lock 2026-02-11T23:30:26.260+0000 7fac683ea640 0 osd.0 49 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:26.260+0000 7fac683ea640 20 osd.0 49 tick_without_osd_lock max_waiting_epoch 50, requesting new map 2026-02-11T23:30:26.260+0000 7fac683ea640 10 monclient: _renew_subs 2026-02-11T23:30:26.260+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:26.260+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=50}) -- 0x44a6680 con 0x396bc00 2026-02-11T23:30:26.260+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:26.260+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:26.260+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:26.260+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.152 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:26.260+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:26.261, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:26.260+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:26.260+0000 7fac683ea640 10 osd.0 49 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:26.260+0000 7fac683ea640 20 osd.0 49 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:26.260+0000 7fac683ea640 10 osd.0 49 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:26.260+0000 7fac683ea640 20 osd.0 49 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:26.260+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 33 ==== osd_map(50..51 src has 1..51) ==== 1200+0+0 (crc 0 0 0) 0x46ea700 con 0x396bc00 2026-02-11T23:30:26.260+0000 7fac626e8640 20 osd.0 49 OSD::ms_dispatch: osd_map(50..51 src has 1..51) 2026-02-11T23:30:26.260+0000 7fac626e8640 20 osd.0 49 _dispatch 0x46ea700 osd_map(50..51 src has 1..51) 2026-02-11T23:30:26.260+0000 7fac626e8640 3 osd.0 49 handle_osd_map epochs [50,51], i have 49, src has [1,51] 2026-02-11T23:30:26.260+0000 7fac626e8640 10 osd.0 49 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:26.260+0000 7fac626e8640 10 osd.0 49 handle_osd_map got inc map for epoch 50 2026-02-11T23:30:26.260+0000 7fac626e8640 20 osd.0 49 got_full_map 50, nothing requested 2026-02-11T23:30:26.260+0000 7fac626e8640 10 osd.0 49 handle_osd_map got inc map for epoch 51 2026-02-11T23:30:26.260+0000 7fac626e8640 20 osd.0 49 got_full_map 51, nothing requested 2026-02-11T23:30:26.260+0000 7fac626e8640 20 osd.0 49 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:26.260+0000 7fac626e8640 20 osd.0 49 handle_osd_map pg_num_history pg_num_history(e51 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1}} deleted_pools ) 2026-02-11T23:30:26.260+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {50={},51={}} 2026-02-11T23:30:26.260+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:26.260+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e51 maps [1~51] lci=[5,51] tlb=1) 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0 49 _committed_osd_maps 50..51 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0 49 advance to epoch 50 (<= last 51 <= newest_map 51) 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0 50 advance to epoch 51 (<= last 51 <= newest_map 51) 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0 51 consume_map version 51 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0 49 await_reserved_maps epoch:51 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0 49 await_reserved_maps done 0 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0 51 consume_map 51 -- publish done 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0 51 identify_splits_and_merges 4.0 e49 to e51 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0 51 identify_splits_and_merges 4.0 e49 to e51 pg_nums {45=1} 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0 51 identify_splits_and_merges 1.0 e49 to e51 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0 51 identify_splits_and_merges 1.0 e49 to e51 pg_nums {24=1} 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0 51 prune_pg_created 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0:0.consume_map 51 (was 49) 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 50 <= 51, requeueing 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {50=} 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0:1.consume_map 51 (was 49) 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0:2.consume_map 51 (was 49) 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0:3.consume_map 51 (was 49) 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0:4.consume_map 51 (was 49) 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0:5.consume_map 51 (was 49) 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0:6.consume_map 51 (was 49) 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0:7.consume_map 51 (was 49) 2026-02-11T23:30:26.260+0000 7fac5e983640 10 osd.0 51 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.260+0000 7fac5e983640 15 osd.0 51 enqueue_peering_evt 4.0 epoch_sent: 51 epoch_requested: 51 NullEvt 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 51 epoch_requested: 51 NullEvt) class_id 2 prio 255 cost 10 e51) 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) pg 0x4856000 2026-02-11T23:30:26.260+0000 7fac5e983640 15 osd.0 51 enqueue_peering_evt 1.0 epoch_sent: 51 epoch_requested: 51 NullEvt 2026-02-11T23:30:26.260+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 51 epoch_requested: 51 NullEvt) class_id 2 prio 255 cost 10 e51) 2026-02-11T23:30:26.260+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.260+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:26.260+0000 7fac5e983640 7 osd.0 51 activate_map version 51 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] handle_advance_map: 50 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 49 -> 50 2026-02-11T23:30:26.260+0000 7fac55971640 20 osd.0 51 agent_entry empty queue 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] state: Started advmap 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] hit_set_clear 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] agent_stop 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 51 advance_pg new pool opts old pool opts 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] handle_advance_map: 51 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 50 -> 51 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 50 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] state: Started advmap 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] hit_set_clear 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] agent_stop 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 51 advance_pg new pool opts old pool opts 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] handle_activate_map: epoch range: 50..51 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 49 while current is 51 2026-02-11T23:30:26.260+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] do_peering_event: epoch_sent: 50 epoch_requested: 50 MLease epoch 50 from osd.4 pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s) 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::proc_lease pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s) 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.886684418s -> ru 34.704032898s 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::proc_lease readable_until now 34.704032898s 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.887022972s -> ruub 34.991786957s 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::proc_lease readable_until_ub now 34.991786957s 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 51 get_con_osd_cluster to osd.4 from_epoch 51 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 51 get_nextmap_reserved map_reservations: {51=1} 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 51 release_map epoch: 51 2026-02-11T23:30:26.260+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_lease_ack(4.0 pg_lease_ack(ruub 25.104763031s) e51/51) -- 0x25d9080 con 0x3e4a000 2026-02-11T23:30:26.260+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.260+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.260+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) pg 0x4856000 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) queued 2026-02-11T23:30:26.260+0000 7fac55170640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=45/45 les/c/f=46/46/0 sis=48 pruub=14.900403976s) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] do_peering_event: epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s) 2026-02-11T23:30:26.260+0000 7fac55170640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active pruub 33.891853333s@ mbc={}] PeeringState::update_history advanced history from ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48 2026-02-11T23:30:26.260+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.260+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.260+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 51 epoch_requested: 51 NullEvt) class_id 2immediate prio 255 cost 10 e51) queued 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e50) pg 0x4856000 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/45 les/c/f=49/46/0 sis=48) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 50 epoch_requested: 50 MInfoRec from 4 info: 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) pg_lease(ru 24.817348480s ub 25.104763031s int 16.000000000s) 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 pi=[45,48)/1 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:26.260+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.260+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.260+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 51 epoch_requested: 51 NullEvt) class_id 2immediate prio 255 cost 10 e51) pg 0x4856000 2026-02-11T23:30:26.260+0000 7fac55170640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 51 epoch_requested: 51 NullEvt 2026-02-11T23:30:26.260+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 51 epoch_requested: 51 NullEvt) class_id 2immediate prio 255 cost 10 e51) queued 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 51 epoch_requested: 51 NullEvt) class_id 2immediate prio 255 cost 10 e51) pg 0x40f8000 2026-02-11T23:30:26.260+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 50 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 49 -> 50 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 49 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 51 advance_pg new pool opts old pool opts 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 51 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 50 -> 51 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 50 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 51 advance_pg new pool opts old pool opts 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 50..51 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 51 2026-02-11T23:30:26.260+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.260+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 51 epoch_requested: 51 NullEvt 2026-02-11T23:30:26.260+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.260+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.261+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.261+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.261+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.261+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.579+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 11 ==== pg_query2(5.0 5.0 query(info 0'0 epoch_sent 52) e52/52) ==== 158+0+0 (unknown 0 0 0) 0x3f3f0e0 con 0x3741000 2026-02-11T23:30:26.579+0000 7fac723fe640 15 osd.0 51 enqueue_peering_evt 5.0 epoch_sent: 52 epoch_requested: 52 MQuery 5.0 from 3 query_epoch 52 query: query(info 0'0 epoch_sent 52) 2026-02-11T23:30:26.579+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 5.0 from 3 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2 prio 255 cost 10 e52) 2026-02-11T23:30:26.579+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.579+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 5.0 from 3 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2immediate prio 255 cost 10 e52) queued 2026-02-11T23:30:26.579+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.579+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 5.0 from 3 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2immediate prio 255 cost 10 e52) pg 0 2026-02-11T23:30:26.579+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 map 52 > 51 2026-02-11T23:30:26.579+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 5.0 peering, item epoch is 52, will wait on OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 5.0 from 3 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2immediate prio 255 cost 10 e52) 2026-02-11T23:30:26.579+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 12 ==== osd_map(48..52 src has 1..52) ==== 2971+0+0 (unknown 0 0 0) 0x46ea540 con 0x3741000 2026-02-11T23:30:26.579+0000 7fac606e4640 20 osd.0 51 OSD::ms_dispatch: osd_map(48..52 src has 1..52) 2026-02-11T23:30:26.579+0000 7fac606e4640 20 osd.0 51 _dispatch 0x46ea540 osd_map(48..52 src has 1..52) 2026-02-11T23:30:26.579+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.579+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.579+0000 7fac606e4640 3 osd.0 51 handle_osd_map epochs [48,52], i have 51, src has [1,52] 2026-02-11T23:30:26.579+0000 7fac606e4640 10 osd.0 51 handle_osd_map: received 0 full maps and 5 incremental maps 2026-02-11T23:30:26.579+0000 7fac606e4640 10 osd.0 51 handle_osd_map got inc map for epoch 52 2026-02-11T23:30:26.579+0000 7fac606e4640 20 osd.0 51 got_full_map 52, nothing requested 2026-02-11T23:30:26.579+0000 7fac606e4640 10 osd.0 51 _track_pools_and_pg_num_changes recording new pool 5 pg_num 1 2026-02-11T23:30:26.579+0000 7fac606e4640 20 osd.0 51 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:26.579+0000 7fac606e4640 20 osd.0 51 handle_osd_map pg_num_history pg_num_history(e52 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1}} deleted_pools ) 2026-02-11T23:30:26.579+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {52={}} 2026-02-11T23:30:26.579+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:26.579+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e52 maps [1~52] lci=[5,52] tlb=1) 2026-02-11T23:30:26.580+0000 7fac5e983640 10 osd.0 51 _committed_osd_maps 52..52 2026-02-11T23:30:26.580+0000 7fac5e983640 10 osd.0 51 advance to epoch 52 (<= last 52 <= newest_map 52) 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0 52 consume_map version 52 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0 51 await_reserved_maps epoch:52 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0 51 await_reserved_maps done 0 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0 52 consume_map 52 -- publish done 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:5.0slot:0x25d8b00 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 5.0 has no pg and waiting_for_split 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0 52 identify_splits_and_merges 4.0 e51 to e52 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0 52 identify_splits_and_merges 4.0 e51 to e52 pg_nums {45=1} 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0 52 identify_splits_and_merges 1.0 e51 to e52 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0 52 identify_splits_and_merges 1.0 e51 to e52 pg_nums {24=1} 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0 52 prune_pg_created 2026-02-11T23:30:26.580+0000 7fac5e983640 10 osd.0:0.consume_map 52 (was 51) 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 5.0 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 5.0 pending_peering first epoch 52 <= 52, requeueing 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 5.0 to_process <> waiting <> waiting_peering {52=} 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:26.580+0000 7fac5e983640 10 osd.0:1.consume_map 52 (was 51) 2026-02-11T23:30:26.580+0000 7fac5e983640 10 osd.0:2.consume_map 52 (was 51) 2026-02-11T23:30:26.580+0000 7fac5e983640 10 osd.0:3.consume_map 52 (was 51) 2026-02-11T23:30:26.580+0000 7fac5e983640 10 osd.0:4.consume_map 52 (was 51) 2026-02-11T23:30:26.580+0000 7fac5e983640 10 osd.0:5.consume_map 52 (was 51) 2026-02-11T23:30:26.580+0000 7fac5e983640 10 osd.0:6.consume_map 52 (was 51) 2026-02-11T23:30:26.580+0000 7fac5e983640 10 osd.0:7.consume_map 52 (was 51) 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 5.0 from 3 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2immediate prio 255 cost 10 e52) queued 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 MQuery 5.0 from 3 query_epoch 52 query: query(info 0'0 epoch_sent 52)) class_id 2immediate prio 255 cost 10 e52) pg 0 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 52 handle_pg_query_nopg 5.0 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 52 pg 5.0 dne 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 52 get_con_osd_cluster to osd.3 from_epoch 52 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 52 get_nextmap_reserved map_reservations: {52=1} 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 52 release_map epoch: 52 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 52 maybe_share_map: con v2:127.0.0.1:6828/1114810440 map epoch 47 -> 52 (shared) 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 52 send_incremental_map epoch range: (48, 52] to v2:127.0.0.1:6828/1114810440 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 52 add_map_inc_bl 49 582 bytes 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 52 add_map_inc_bl 50 574 bytes 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 52 add_map_inc_bl 51 574 bytes 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 52 add_map_inc_bl 52 591 bytes 2026-02-11T23:30:26.580+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- osd_map(48..52 src has 1..52) -- 0x3941a40 con 0x3741000 2026-02-11T23:30:26.580+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_notify2(5.0 5.0 (query:52 sent:52 5.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e52/52) -- 0x4869500 con 0x3741000 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.580+0000 7fac5e983640 10 osd.0 52 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:26.580+0000 7fac5e983640 15 osd.0 52 enqueue_peering_evt 4.0 epoch_sent: 52 epoch_requested: 52 NullEvt 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2 prio 255 cost 10 e52) 2026-02-11T23:30:26.580+0000 7fac5e983640 15 osd.0 52 enqueue_peering_evt 1.0 epoch_sent: 52 epoch_requested: 52 NullEvt 2026-02-11T23:30:26.580+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2 prio 255 cost 10 e52) 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2immediate prio 255 cost 10 e52) queued 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2immediate prio 255 cost 10 e52) pg 0x4856000 2026-02-11T23:30:26.580+0000 7fac5e983640 7 osd.0 52 activate_map version 52 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 52 2026-02-11T23:30:26.580+0000 7fac55971640 20 osd.0 52 agent_entry empty queue 2026-02-11T23:30:26.580+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.580+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2immediate prio 255 cost 10 e52) queued 2026-02-11T23:30:26.580+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.580+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 52 epoch_requested: 52 NullEvt) class_id 2immediate prio 255 cost 10 e52) pg 0x40f8000 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 51 -> 52 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 pg_epoch: 51 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:26.580+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 52 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:26.580+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 51 -> 52 2026-02-11T23:30:26.580+0000 7fac51168640 10 osd.0 pg_epoch: 51 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 52 advance_pg new pool opts old pool opts 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 52..52 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:26.580+0000 7fac51168640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 52 2026-02-11T23:30:26.580+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.580+0000 7fac51168640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:26.580+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 52 epoch_requested: 52 NullEvt 2026-02-11T23:30:26.580+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.580+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.580+0000 7fac51168640 20 osd.0 52 advance_pg new pool opts old pool opts 2026-02-11T23:30:26.580+0000 7fac51168640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 52..52 2026-02-11T23:30:26.580+0000 7fac51168640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:26.580+0000 7fac51168640 20 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 52 2026-02-11T23:30:26.580+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.580+0000 7fac51168640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 52 epoch_requested: 52 NullEvt 2026-02-11T23:30:26.580+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.580+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.630+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 34 ==== osd_map(52..53 src has 1..53) ==== 895+0+0 (crc 0 0 0) 0x3941a40 con 0x396bc00 2026-02-11T23:30:26.630+0000 7fac626e8640 20 osd.0 52 OSD::ms_dispatch: osd_map(52..53 src has 1..53) 2026-02-11T23:30:26.630+0000 7fac626e8640 20 osd.0 52 _dispatch 0x3941a40 osd_map(52..53 src has 1..53) 2026-02-11T23:30:26.630+0000 7fac626e8640 3 osd.0 52 handle_osd_map epochs [52,53], i have 52, src has [1,53] 2026-02-11T23:30:26.630+0000 7fac626e8640 10 osd.0 52 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:26.630+0000 7fac626e8640 10 osd.0 52 handle_osd_map got inc map for epoch 53 2026-02-11T23:30:26.630+0000 7fac626e8640 20 osd.0 52 got_full_map 53, nothing requested 2026-02-11T23:30:26.630+0000 7fac626e8640 20 osd.0 52 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:26.630+0000 7fac626e8640 20 osd.0 52 handle_osd_map pg_num_history pg_num_history(e53 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1}} deleted_pools ) 2026-02-11T23:30:26.630+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {53={}} 2026-02-11T23:30:26.630+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:26.630+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e53 maps [1~53] lci=[5,53] tlb=1) 2026-02-11T23:30:26.630+0000 7fac5e983640 10 osd.0 52 _committed_osd_maps 53..53 2026-02-11T23:30:26.630+0000 7fac5e983640 10 osd.0 52 advance to epoch 53 (<= last 53 <= newest_map 53) 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0 53 consume_map version 53 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0 52 await_reserved_maps epoch:53 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0 52 await_reserved_maps done 0 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0 53 consume_map 53 -- publish done 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:5.0slot:0x25d8b00 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 5.0 has no pg and waiting_for_split 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0 53 identify_splits_and_merges 4.0 e52 to e53 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0 53 identify_splits_and_merges 4.0 e52 to e53 pg_nums {45=1} 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0 53 identify_splits_and_merges 1.0 e52 to e53 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0 53 identify_splits_and_merges 1.0 e52 to e53 pg_nums {24=1} 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0 53 prune_pg_created 2026-02-11T23:30:26.630+0000 7fac5e983640 10 osd.0:0.consume_map 53 (was 52) 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 5.0 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 5.0 empty, pruning 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:26.630+0000 7fac5e983640 10 osd.0:1.consume_map 53 (was 52) 2026-02-11T23:30:26.630+0000 7fac5e983640 10 osd.0:2.consume_map 53 (was 52) 2026-02-11T23:30:26.630+0000 7fac5e983640 10 osd.0:3.consume_map 53 (was 52) 2026-02-11T23:30:26.630+0000 7fac5e983640 10 osd.0:4.consume_map 53 (was 52) 2026-02-11T23:30:26.630+0000 7fac5e983640 10 osd.0:5.consume_map 53 (was 52) 2026-02-11T23:30:26.630+0000 7fac5e983640 10 osd.0:6.consume_map 53 (was 52) 2026-02-11T23:30:26.630+0000 7fac5e983640 10 osd.0:7.consume_map 53 (was 52) 2026-02-11T23:30:26.630+0000 7fac5e983640 10 osd.0 53 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:26.630+0000 7fac5e983640 15 osd.0 53 enqueue_peering_evt 4.0 epoch_sent: 53 epoch_requested: 53 NullEvt 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:26.630+0000 7fac5e983640 15 osd.0 53 enqueue_peering_evt 1.0 epoch_sent: 53 epoch_requested: 53 NullEvt 2026-02-11T23:30:26.630+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:26.630+0000 7fac5e983640 7 osd.0 53 activate_map version 53 2026-02-11T23:30:26.630+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.630+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:26.630+0000 7fac55971640 20 osd.0 53 agent_entry empty queue 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2immediate prio 255 cost 10 e53) pg 0x4856000 2026-02-11T23:30:26.631+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.631+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 53 2026-02-11T23:30:26.631+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.631+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 NullEvt) class_id 2immediate prio 255 cost 10 e53) pg 0x40f8000 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 52 -> 53 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 53 advance_pg new pool opts old pool opts 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 53..53 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 53 2026-02-11T23:30:26.631+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 NullEvt 2026-02-11T23:30:26.631+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.631+0000 7fac51168640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 53 2026-02-11T23:30:26.631+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 52 -> 53 2026-02-11T23:30:26.631+0000 7fac51168640 10 osd.0 pg_epoch: 52 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:26.631+0000 7fac51168640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:26.631+0000 7fac51168640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:26.631+0000 7fac51168640 20 osd.0 53 advance_pg new pool opts old pool opts 2026-02-11T23:30:26.631+0000 7fac51168640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 53..53 2026-02-11T23:30:26.631+0000 7fac51168640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:26.631+0000 7fac51168640 20 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 53 2026-02-11T23:30:26.631+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.631+0000 7fac51168640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 NullEvt 2026-02-11T23:30:26.631+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:26.631+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.631+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 13 ==== PGlog(5.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.157077789s int 16.000000000s) e53/52) ==== 1172+0+0 (unknown 0 0 0) 0x3902000 con 0x3741000 2026-02-11T23:30:26.631+0000 7fac723fe640 15 osd.0 53 enqueue_peering_evt 5.0 epoch_sent: 53 epoch_requested: 52 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.157077789s int 16.000000000s) +create_info 2026-02-11T23:30:26.631+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 52 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.157077789s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 52 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.157077789s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 52 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.157077789s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e53) pg 0 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 no pg, should create on OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 52 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.157077789s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e53) 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 53 _make_pg 5.0 2026-02-11T23:30:26.631+0000 7fac55170640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:5, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:30:26.631+0000 7fac55170640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:5, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:30:26.631+0000 7fac55170640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:5, shard:-1, shard_prefix: , prefixes: {"0000000000000005."}] update_bits prefix updated 2026-02-11T23:30:26.631+0000 7fac55170640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:5, shard:-1, shard_prefix: , prefixes: {"0000000000000005."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:30:26.631+0000 7fac55170640 5 osd.0 pg_epoch: 52 pg[5.0(unlocked)] enter Initial 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 pg_epoch: 52 pg[5.0(unlocked)] enter NotTrimming 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role 2 up [3,4,0] acting [3,4,0] history ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52 past_intervals ([0,0] all_participants={} intervals=[]) 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [3,4,0]+[3,4,0] 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x38637a0] 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 19.467599869s + 0.000000000s) 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:26.631+0000 7fac55170640 15 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:26.631+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:26.631+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:26.631+0000 7fac55170640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:30:26.631+0000 7fac55170640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:30:26.631+0000 7fac55170640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:30:26.631+0000 7fac55170640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:30:26.631+0000 7fac55170640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:30:26.631+0000 7fac55170640 10 end of _write_log_and_missing 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:30:26.631+0000 7fac55170640 5 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] exit Initial 0.000061 0 0.000000 2026-02-11T23:30:26.631+0000 7fac55170640 5 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 52 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 52..52 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:26.631+0000 7fac55170640 5 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] exit Reset 0.000010 1 0.000010 2026-02-11T23:30:26.631+0000 7fac55170640 5 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:30:26.631+0000 7fac55170640 5 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:30:26.631+0000 7fac55170640 1 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:30:26.631+0000 7fac55170640 5 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:26.631+0000 7fac55170640 5 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 52 while current is 52 2026-02-11T23:30:26.631+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 53 handle_pg_create_info new pg pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0:0._attach_pg 5.0 0x46f4000 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0:0._wake_pg_slot _wake_pg_slot 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 53 identify_splits_and_merges 5.0 e52 to e53 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 53 identify_splits_and_merges 5.0 e52 to e53 pg_nums {52=1} 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0:0._prime_splits 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] handle_advance_map: 53 2026-02-11T23:30:26.631+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0:0.update_pg_epoch 5.0 52 -> 53 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 52 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [3,4,0]/[3,4,0] -- 3/3 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 53 advance_pg new pool opts old pool opts 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 53..53 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 52 while current is 53 2026-02-11T23:30:26.631+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 52 MLogRec from 3 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 28.157077789s int 16.000000000s) +create_info 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] state: got info+log from osd.3 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:26.631+0000 7fac55170640 10 merge_log log((0'0,0'0], crt=0'0) from osd.3 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:26.631+0000 7fac55170640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:26.631+0000 7fac55170640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:26.631+0000 7fac55170640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 28.157077789s int 16.000000000s) 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] PeeringState::proc_lease peer_clock_delta_lb -7.310479641s -> ruub 35.467559814s 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] PeeringState::proc_lease readable_until_ub now 35.467559814s 2026-02-11T23:30:26.631+0000 7fac55170640 5 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] exit Started/Stray 0.000054 3 0.000010 2026-02-11T23:30:26.631+0000 7fac55170640 5 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:26.631+0000 7fac55170640 5 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=0/0 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:26.631+0000 7fac55170640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:26.631+0000 7fac55170640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:26.631+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:30:26.631+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.631+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.632+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.632+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.632+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.632+0000 7fac55170640 15 osd.0 53 enqueue_peering_evt 5.0 epoch_sent: 53 epoch_requested: 53 ActivateCommitted from 53 processed at 53 2026-02-11T23:30:26.632+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 ActivateCommitted from 53 processed at 53) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:26.632+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:26.632+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 ActivateCommitted from 53 processed at 53) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:26.632+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:26.632+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 ActivateCommitted from 53 processed at 53) class_id 2immediate prio 255 cost 10 e53) pg 0x46f4000 2026-02-11T23:30:26.632+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 ActivateCommitted from 53 processed at 53 2026-02-11T23:30:26.632+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:26.632+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 unknown mbc={}] state: react 53 telling primary 2026-02-11T23:30:26.632+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:26.632+0000 7fac55170640 20 osd.0 53 get_con_osd_cluster to osd.3 from_epoch 53 2026-02-11T23:30:26.632+0000 7fac55170640 20 osd.0 53 get_nextmap_reserved map_reservations: {53=1} 2026-02-11T23:30:26.632+0000 7fac55170640 20 osd.0 53 release_map epoch: 53 2026-02-11T23:30:26.632+0000 7fac55170640 10 osd.0 53 maybe_share_map: con v2:127.0.0.1:6828/1114810440 map epoch 52 -> 53 (shared) 2026-02-11T23:30:26.632+0000 7fac55170640 10 osd.0 53 send_incremental_map epoch range: (53, 53] to v2:127.0.0.1:6828/1114810440 2026-02-11T23:30:26.632+0000 7fac55170640 10 osd.0 53 add_map_inc_bl 53 252 bytes 2026-02-11T23:30:26.632+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- osd_map(53..53 src has 1..53) -- 0x38e21c0 con 0x3741000 2026-02-11T23:30:26.632+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_info2(5.0 5.0 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/0 les/c/f=53/0/0 sis=52) e53/53) -- 0x4869c00 con 0x3741000 2026-02-11T23:30:26.632+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.051+0000 7fac68beb640 10 osd.0 53 tick 2026-02-11T23:30:27.051+0000 7fac68beb640 20 osd.0 53 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:27.089+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 14 ==== pg_lease(5.0 pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s) e53/53) ==== 58+0+0 (unknown 0 0 0) 0x3b1bb00 con 0x3741000 2026-02-11T23:30:27.089+0000 7fac723fe640 15 osd.0 53 enqueue_peering_evt 5.0 epoch_sent: 53 epoch_requested: 53 MLease epoch 53 from osd.3 pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s) 2026-02-11T23:30:27.089+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MLease epoch 53 from osd.3 pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s)) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MLease epoch 53 from osd.3 pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MLease epoch 53 from osd.3 pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) pg 0x46f4000 2026-02-11T23:30:27.089+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 MLease epoch 53 from osd.3 pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s) 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.089+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s) 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -7.310237408s -> ru 35.467311859s 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 35.467311859s 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -7.310479641s -> ruub 35.925540924s 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 35.925540924s 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 53 get_con_osd_cluster to osd.3 from_epoch 53 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 53 get_nextmap_reserved map_reservations: {53=1} 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 53 release_map epoch: 53 2026-02-11T23:30:27.089+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_lease_ack(5.0 pg_lease_ack(ruub 28.615062714s) e53/53) -- 0x25d9600 con 0x3741000 2026-02-11T23:30:27.089+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 15 ==== pg_info2(5.0 5.0 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/0 les/c/f=53/0/0 sis=52) e53/53) ==== 1126+0+0 (unknown 0 0 0) 0x4869c00 con 0x3741000 2026-02-11T23:30:27.089+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.089+0000 7fac723fe640 15 osd.0 53 enqueue_peering_evt 5.0 epoch_sent: 53 epoch_requested: 53 MInfoRec from 3 info: 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/0 les/c/f=53/0/0 sis=52) pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s) 2026-02-11T23:30:27.089+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 3 info: 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/0 les/c/f=53/0/0 sis=52) pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s)) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 3 info: 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/0 les/c/f=53/0/0 sis=52) pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 3 info: 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/0 les/c/f=53/0/0 sis=52) pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) pg 0x46f4000 2026-02-11T23:30:27.089+0000 7fac51168640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=0/0 les/c/f=0/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 MInfoRec from 3 info: 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/0 les/c/f=53/0/0 sis=52) pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s) 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/0 les/c/f=53/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=52/52 lis/c=52/0 les/c/f=53/0/0 sis=52 2026-02-11T23:30:27.089+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.089+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 16 ==== pg_info2(5.0 5.0 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) e53/53) ==== 1126+0+0 (unknown 0 0 0) 0x4869c00 con 0x3741000 2026-02-11T23:30:27.089+0000 7fac723fe640 15 osd.0 53 enqueue_peering_evt 5.0 epoch_sent: 53 epoch_requested: 53 MInfoRec from 3 info: 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s) 2026-02-11T23:30:27.089+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 3 info: 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s)) class_id 2 prio 255 cost 10 e53) 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 3 info: 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) queued 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 53 epoch_requested: 53 MInfoRec from 3 info: 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e53) pg 0x46f4000 2026-02-11T23:30:27.089+0000 7fac51168640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/0 les/c/f=53/0/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 53 epoch_requested: 53 MInfoRec from 3 info: 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) pg_lease(ru 28.157077789s ub 28.615062714s int 16.000000000s) 2026-02-11T23:30:27.089+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:27.089+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:27.089+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.090+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.090+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.140+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 35 ==== osd_map(53..54 src has 1..54) ==== 878+0+0 (crc 0 0 0) 0x38e21c0 con 0x396bc00 2026-02-11T23:30:27.140+0000 7fac626e8640 20 osd.0 53 OSD::ms_dispatch: osd_map(53..54 src has 1..54) 2026-02-11T23:30:27.140+0000 7fac626e8640 20 osd.0 53 _dispatch 0x38e21c0 osd_map(53..54 src has 1..54) 2026-02-11T23:30:27.140+0000 7fac626e8640 3 osd.0 53 handle_osd_map epochs [53,54], i have 53, src has [1,54] 2026-02-11T23:30:27.140+0000 7fac626e8640 10 osd.0 53 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:27.140+0000 7fac626e8640 10 osd.0 53 handle_osd_map got inc map for epoch 54 2026-02-11T23:30:27.140+0000 7fac626e8640 20 osd.0 53 got_full_map 54, nothing requested 2026-02-11T23:30:27.140+0000 7fac626e8640 20 osd.0 53 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:27.140+0000 7fac626e8640 20 osd.0 53 handle_osd_map pg_num_history pg_num_history(e54 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1}} deleted_pools ) 2026-02-11T23:30:27.140+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {54={}} 2026-02-11T23:30:27.140+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:27.140+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e54 maps [1~54] lci=[5,54] tlb=1) 2026-02-11T23:30:27.140+0000 7fac5e983640 10 osd.0 53 _committed_osd_maps 54..54 2026-02-11T23:30:27.140+0000 7fac5e983640 10 osd.0 53 advance to epoch 54 (<= last 54 <= newest_map 54) 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 54 consume_map version 54 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 53 await_reserved_maps epoch:54 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 53 await_reserved_maps done 0 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 54 consume_map 54 -- publish done 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:5.0slot:0x25809a0 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 54 identify_splits_and_merges 5.0 e53 to e54 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 54 identify_splits_and_merges 5.0 e53 to e54 pg_nums {52=1} 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 54 identify_splits_and_merges 4.0 e53 to e54 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 54 identify_splits_and_merges 4.0 e53 to e54 pg_nums {45=1} 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 54 identify_splits_and_merges 1.0 e53 to e54 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 54 identify_splits_and_merges 1.0 e53 to e54 pg_nums {24=1} 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 54 prune_pg_created 2026-02-11T23:30:27.140+0000 7fac5e983640 10 osd.0:0.consume_map 54 (was 53) 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 5.0 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:27.140+0000 7fac5e983640 10 osd.0:1.consume_map 54 (was 53) 2026-02-11T23:30:27.140+0000 7fac5e983640 10 osd.0:2.consume_map 54 (was 53) 2026-02-11T23:30:27.140+0000 7fac5e983640 10 osd.0:3.consume_map 54 (was 53) 2026-02-11T23:30:27.140+0000 7fac5e983640 10 osd.0:4.consume_map 54 (was 53) 2026-02-11T23:30:27.140+0000 7fac5e983640 10 osd.0:5.consume_map 54 (was 53) 2026-02-11T23:30:27.140+0000 7fac5e983640 10 osd.0:6.consume_map 54 (was 53) 2026-02-11T23:30:27.140+0000 7fac5e983640 10 osd.0:7.consume_map 54 (was 53) 2026-02-11T23:30:27.140+0000 7fac5e983640 10 osd.0 54 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:27.140+0000 7fac5e983640 15 osd.0 54 enqueue_peering_evt 5.0 epoch_sent: 54 epoch_requested: 54 NullEvt 2026-02-11T23:30:27.140+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2 prio 255 cost 10 e54) 2026-02-11T23:30:27.141+0000 7fac5e983640 15 osd.0 54 enqueue_peering_evt 4.0 epoch_sent: 54 epoch_requested: 54 NullEvt 2026-02-11T23:30:27.141+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2 prio 255 cost 10 e54) 2026-02-11T23:30:27.141+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:27.141+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2immediate prio 255 cost 10 e54) queued 2026-02-11T23:30:27.141+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:27.141+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2immediate prio 255 cost 10 e54) pg 0x46f4000 2026-02-11T23:30:27.141+0000 7fac5e983640 15 osd.0 54 enqueue_peering_evt 1.0 epoch_sent: 54 epoch_requested: 54 NullEvt 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2immediate prio 255 cost 10 e54) queued 2026-02-11T23:30:27.141+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2 prio 255 cost 10 e54) 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2immediate prio 255 cost 10 e54) pg 0x4856000 2026-02-11T23:30:27.141+0000 7fac5e983640 7 osd.0 54 activate_map version 54 2026-02-11T23:30:27.141+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 54 2026-02-11T23:30:27.141+0000 7fac55170640 20 osd.0:0.update_pg_epoch 5.0 53 -> 54 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 54 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 53 -> 54 2026-02-11T23:30:27.141+0000 7fac55170640 10 osd.0 pg_epoch: 53 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [3,4,0]/[3,4,0] -- 3/3 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 53 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:27.141+0000 7fac55971640 20 osd.0 54 agent_entry empty queue 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:27.141+0000 7fac55170640 10 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 54 advance_pg new pool opts old pool opts 2026-02-11T23:30:27.141+0000 7fac55170640 10 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 54..54 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:27.141+0000 7fac55170640 10 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 54 2026-02-11T23:30:27.141+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:27.141+0000 7fac55170640 20 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 54 epoch_requested: 54 NullEvt 2026-02-11T23:30:27.141+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:27.141+0000 7fac55170640 20 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2immediate prio 255 cost 10 e54) queued 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 54 epoch_requested: 54 NullEvt) class_id 2immediate prio 255 cost 10 e54) pg 0x40f8000 2026-02-11T23:30:27.141+0000 7fac55170640 20 osd.0 54 advance_pg new pool opts old pool opts 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 54 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 53 -> 54 2026-02-11T23:30:27.141+0000 7fac55170640 10 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] handle_activate_map: epoch range: 54..54 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 53 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:27.141+0000 7fac55170640 10 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:27.141+0000 7fac55170640 20 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 54 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 54 advance_pg new pool opts old pool opts 2026-02-11T23:30:27.141+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 54..54 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:27.141+0000 7fac55170640 10 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 54 2026-02-11T23:30:27.141+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:27.141+0000 7fac55170640 10 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 54 epoch_requested: 54 NullEvt 2026-02-11T23:30:27.141+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 54 epoch_requested: 54 NullEvt 2026-02-11T23:30:27.141+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:27.141+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:27.141+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.141+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:27.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:27.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:57.168437+0000) 2026-02-11T23:30:27.237+0000 7fac683ea640 10 osd.0 54 tick_without_osd_lock 2026-02-11T23:30:27.237+0000 7fac683ea640 0 osd.0 54 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:27.237+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:27.237+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:27.237+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:27.237+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.152 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:27.237+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:27.238, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:27.237+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:27.237+0000 7fac683ea640 10 osd.0 54 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:27.237+0000 7fac683ea640 20 osd.0 54 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:27.237+0000 7fac683ea640 10 osd.0 54 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:27.237+0000 7fac683ea640 20 osd.0 54 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:27.664+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 17 ==== pg_query2(5.0 5.0 query(info 0'0 epoch_sent 55) e55/55) ==== 158+0+0 (unknown 0 0 0) 0x3f3f0e0 con 0x3741000 2026-02-11T23:30:27.664+0000 7fac723fe640 15 osd.0 54 enqueue_peering_evt 5.0 epoch_sent: 55 epoch_requested: 55 MQuery 5.0 from 3 query_epoch 55 query: query(info 0'0 epoch_sent 55) 2026-02-11T23:30:27.664+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 MQuery 5.0 from 3 query_epoch 55 query: query(info 0'0 epoch_sent 55)) class_id 2 prio 255 cost 10 e55) 2026-02-11T23:30:27.664+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:27.664+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 MQuery 5.0 from 3 query_epoch 55 query: query(info 0'0 epoch_sent 55)) class_id 2immediate prio 255 cost 10 e55) queued 2026-02-11T23:30:27.664+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:27.664+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 MQuery 5.0 from 3 query_epoch 55 query: query(info 0'0 epoch_sent 55)) class_id 2immediate prio 255 cost 10 e55) pg 0x46f4000 2026-02-11T23:30:27.664+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 5.0 peering, item epoch is 55, will wait on OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 MQuery 5.0 from 3 query_epoch 55 query: query(info 0'0 epoch_sent 55)) class_id 2immediate prio 255 cost 10 e55) 2026-02-11T23:30:27.664+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.664+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 18 ==== osd_map(53..55 src has 1..55) ==== 1460+0+0 (unknown 0 0 0) 0x46ea1c0 con 0x3741000 2026-02-11T23:30:27.664+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.664+0000 7fac606e4640 20 osd.0 54 OSD::ms_dispatch: osd_map(53..55 src has 1..55) 2026-02-11T23:30:27.664+0000 7fac606e4640 20 osd.0 54 _dispatch 0x46ea1c0 osd_map(53..55 src has 1..55) 2026-02-11T23:30:27.664+0000 7fac606e4640 3 osd.0 54 handle_osd_map epochs [53,55], i have 54, src has [1,55] 2026-02-11T23:30:27.664+0000 7fac606e4640 10 osd.0 54 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:27.664+0000 7fac606e4640 10 osd.0 54 handle_osd_map got inc map for epoch 55 2026-02-11T23:30:27.664+0000 7fac606e4640 20 osd.0 54 got_full_map 55, nothing requested 2026-02-11T23:30:27.664+0000 7fac606e4640 20 osd.0 54 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:27.664+0000 7fac606e4640 20 osd.0 54 handle_osd_map pg_num_history pg_num_history(e55 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1}} deleted_pools ) 2026-02-11T23:30:27.664+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {55={}} 2026-02-11T23:30:27.664+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:27.664+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e55 maps [1~55] lci=[5,55] tlb=1) 2026-02-11T23:30:27.665+0000 7fac5e983640 10 osd.0 54 _committed_osd_maps 55..55 2026-02-11T23:30:27.665+0000 7fac5e983640 10 osd.0 54 advance to epoch 55 (<= last 55 <= newest_map 55) 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 55 consume_map version 55 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 54 await_reserved_maps epoch:55 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 54 await_reserved_maps done 0 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 55 consume_map 55 -- publish done 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:5.0slot:0x25809a0 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 55 identify_splits_and_merges 5.0 e54 to e55 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 55 identify_splits_and_merges 5.0 e54 to e55 pg_nums {52=1} 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 55 identify_splits_and_merges 4.0 e54 to e55 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 55 identify_splits_and_merges 4.0 e54 to e55 pg_nums {45=1} 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 55 identify_splits_and_merges 1.0 e54 to e55 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 55 identify_splits_and_merges 1.0 e54 to e55 pg_nums {24=1} 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 55 prune_pg_created 2026-02-11T23:30:27.665+0000 7fac5e983640 10 osd.0:0.consume_map 55 (was 54) 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 5.0 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 5.0 pending_peering first epoch 55 <= 55, requeueing 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 5.0 to_process <> waiting <> waiting_peering {55=} 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:27.665+0000 7fac5e983640 10 osd.0:1.consume_map 55 (was 54) 2026-02-11T23:30:27.665+0000 7fac5e983640 10 osd.0:2.consume_map 55 (was 54) 2026-02-11T23:30:27.665+0000 7fac5e983640 10 osd.0:3.consume_map 55 (was 54) 2026-02-11T23:30:27.665+0000 7fac5e983640 10 osd.0:4.consume_map 55 (was 54) 2026-02-11T23:30:27.665+0000 7fac5e983640 10 osd.0:5.consume_map 55 (was 54) 2026-02-11T23:30:27.665+0000 7fac5e983640 10 osd.0:6.consume_map 55 (was 54) 2026-02-11T23:30:27.665+0000 7fac5e983640 10 osd.0:7.consume_map 55 (was 54) 2026-02-11T23:30:27.665+0000 7fac5e983640 10 osd.0 55 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 MQuery 5.0 from 3 query_epoch 55 query: query(info 0'0 epoch_sent 55)) class_id 2immediate prio 255 cost 10 e55) queued 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 MQuery 5.0 from 3 query_epoch 55 query: query(info 0'0 epoch_sent 55)) class_id 2immediate prio 255 cost 10 e55) pg 0x46f4000 2026-02-11T23:30:27.665+0000 7fac5e983640 15 osd.0 55 enqueue_peering_evt 5.0 epoch_sent: 55 epoch_requested: 55 NullEvt 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2 prio 255 cost 10 e55) 2026-02-11T23:30:27.665+0000 7fac5e983640 15 osd.0 55 enqueue_peering_evt 4.0 epoch_sent: 55 epoch_requested: 55 NullEvt 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2 prio 255 cost 10 e55) 2026-02-11T23:30:27.665+0000 7fac5e983640 15 osd.0 55 enqueue_peering_evt 1.0 epoch_sent: 55 epoch_requested: 55 NullEvt 2026-02-11T23:30:27.665+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2 prio 255 cost 10 e55) 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2immediate prio 255 cost 10 e55) queued 2026-02-11T23:30:27.665+0000 7fac5e983640 7 osd.0 55 activate_map version 55 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] handle_advance_map: 55 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0:0.update_pg_epoch 5.0 54 -> 55 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 54 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [3,4]/[3,4] -- 3/3 2026-02-11T23:30:27.665+0000 7fac55971640 20 osd.0 55 agent_entry empty queue 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [3,4] newacting [3,4] 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:27.665+0000 7fac55170640 5 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 1.033298 8 0.000077 2026-02-11T23:30:27.665+0000 7fac55170640 5 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] exit Started/ReplicaActive 1.033303 0 0.000000 2026-02-11T23:30:27.665+0000 7fac55170640 5 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] exit Started 1.033365 0 0.000000 2026-02-11T23:30:27.665+0000 7fac55170640 5 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=52 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 55 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=55 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=55 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=55 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=55 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [3,4] newacting [3,4] 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=55 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4,0] r=2 lpr=55 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 55 2026-02-11T23:30:27.665+0000 7fac55170640 15 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4] r=2 lpr=55 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(52-54 up [3, 4, 0](3) acting [3, 4, 0](3)) up_thru 52 up_from 17 last_epoch_clean 53 interval(52-54 up [3, 4, 0](3) acting [3, 4, 0](3) maybe_went_rw) : primary up 17-52 includes interval 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=52) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([52,54] all_participants={0, 3, 4} intervals=[([52,54] acting={0, 3, 4})]) 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [3,4]+[3,4] 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active pruub 35.925540924s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 35.925540924s (mnow 20.501041412s + 15.424501419s) 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active pruub 35.925540924s@ mbc={}] cancel_recovery 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active pruub 35.925540924s@ mbc={}] clear_recovery_state 2026-02-11T23:30:27.665+0000 7fac55170640 15 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active pruub 35.925540924s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active pruub 35.925540924s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:27.665+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active pruub 35.925540924s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active pruub 35.925540924s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active pruub 35.925540924s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:27.665+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:27.665+0000 7fac55170640 1 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 active pruub 35.925540924s@ mbc={}] PeeringState::start_peering_interval up [3,4,0] -> [3,4], acting [3,4,0] -> [3,4], acting_primary 3 -> 3, up_primary 3 -> 3, role 2 -> -1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] clear_primary_state 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] release_backoffs [5:00000000::::0,MAX) 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:27.665+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:27.665+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] agent_stop 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] on_change 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] on_change_cleanup 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] on_change 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] exit NotTrimming 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown pruub 35.925540924s@ mbc={}] enter NotTrimming 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] plpg_on_role_change 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] hit_set_clear 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] agent_stop 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 55 advance_pg new pool opts old pool opts 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] handle_activate_map: epoch range: 55..55 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424501419s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:27.665+0000 7fac55170640 5 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424428940s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] exit Reset 0.000110 1 0.000111 2026-02-11T23:30:27.665+0000 7fac55170640 5 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424428940s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] enter Started 2026-02-11T23:30:27.665+0000 7fac55170640 5 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424428940s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] enter Start 2026-02-11T23:30:27.665+0000 7fac55170640 1 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424428940s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:27.665+0000 7fac55170640 5 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424428940s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] exit Start 0.000003 0 0.000000 2026-02-11T23:30:27.665+0000 7fac55170640 5 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424428940s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] enter Started/Stray 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424428940s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 53 while current is 55 2026-02-11T23:30:27.665+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424428940s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424428940s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] do_peering_event: epoch_sent: 55 epoch_requested: 55 MQuery 5.0 from 3 query_epoch 55 query: query(info 0'0 epoch_sent 55) 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:27.665+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 55 get_con_osd_cluster to osd.3 from_epoch 55 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 55 get_nextmap_reserved map_reservations: {55=1} 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 55 release_map epoch: 55 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 55 maybe_share_map: con v2:127.0.0.1:6828/1114810440 map epoch 53 -> 55 (shared) 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 55 send_incremental_map epoch range: (54, 55] to v2:127.0.0.1:6828/1114810440 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 55 add_map_inc_bl 54 574 bytes 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 55 add_map_inc_bl 55 574 bytes 2026-02-11T23:30:27.665+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- osd_map(54..55 src has 1..55) -- 0x46eac40 con 0x3741000 2026-02-11T23:30:27.665+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_notify2(5.0 5.0 (query:55 sent:55 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424428940s) ([52,54] all_participants={0, 3, 4} intervals=[([52,54] acting={0, 3, 4})])) e55/55) -- 0x486a300 con 0x3741000 2026-02-11T23:30:27.665+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_notify2(5.0 5.0 (query:55 sent:55 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) ([52,54] all_participants={0, 3, 4} intervals=[([52,54] acting={0, 3, 4})])) e55/55) -- 0x486aa00 con 0x3741000 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2immediate prio 255 cost 10 e55) queued 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2immediate prio 255 cost 10 e55) pg 0x4856000 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2immediate prio 255 cost 10 e55) pg 0x46f4000 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 55 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 54 -> 55 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 54 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:27.665+0000 7fac51168640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] do_peering_event: epoch_sent: 55 epoch_requested: 55 NullEvt 2026-02-11T23:30:27.665+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2immediate prio 255 cost 10 e55) queued 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 55 epoch_requested: 55 NullEvt) class_id 2immediate prio 255 cost 10 e55) pg 0x40f8000 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 55 advance_pg new pool opts old pool opts 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 55..55 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 55 2026-02-11T23:30:27.665+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 55 2026-02-11T23:30:27.665+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 54 -> 55 2026-02-11T23:30:27.665+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 55 epoch_requested: 55 NullEvt 2026-02-11T23:30:27.665+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:27.665+0000 7fac51168640 10 osd.0 pg_epoch: 54 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.665+0000 7fac51168640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:27.665+0000 7fac51168640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 55 advance_pg new pool opts old pool opts 2026-02-11T23:30:27.665+0000 7fac51168640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 55..55 2026-02-11T23:30:27.665+0000 7fac51168640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 55 2026-02-11T23:30:27.665+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:27.665+0000 7fac51168640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 55 epoch_requested: 55 NullEvt 2026-02-11T23:30:27.665+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.665+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:27.665+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.009+0000 7fac60ee5640 20 osd.0 55 reports for 0 queries 2026-02-11T23:30:28.009+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x34c3180 con 0x396a800 2026-02-11T23:30:28.009+0000 7fac60ee5640 15 osd.0 55 collect_pg_stats 2026-02-11T23:30:28.009+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836485 v 0) -- 0x489d500 con 0x396a800 2026-02-11T23:30:28.057+0000 7fac68beb640 10 osd.0 55 tick 2026-02-11T23:30:28.057+0000 7fac68beb640 10 osd.0 55 maybe_update_heartbeat_peers forcing update after 20.306812 seconds 2026-02-11T23:30:28.057+0000 7fac68beb640 10 osd.0 55 reset_heartbeat_peers 2026-02-11T23:30:28.057+0000 7fac68beb640 10 osd.0 55 maybe_update_heartbeat_peers updating 2026-02-11T23:30:28.057+0000 7fac68beb640 10 osd.0 55 adding neighbor peer osd.1 2026-02-11T23:30:28.057+0000 7fac68beb640 10 osd.0 55 adding neighbor peer osd.4 2026-02-11T23:30:28.057+0000 7fac68beb640 10 osd.0 55 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:30:28.057+0000 7fac68beb640 20 osd.0 55 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:28.071+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 19 ==== osd pg remove(epoch 57; pg5.0; ) ==== 32+0+0 (unknown 0 0 0) 0x25d8420 con 0x3741000 2026-02-11T23:30:28.071+0000 7fac723fe640 7 osd.0 55 handle_fast_pg_remove osd pg remove(epoch 57; pg5.0; ) from osd.3 2026-02-11T23:30:28.071+0000 7fac723fe640 15 osd.0 55 enqueue_peering_evt 5.0 epoch_sent: 57 epoch_requested: 57 DeleteStart 2026-02-11T23:30:28.071+0000 7fac723fe640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 DeleteStart) class_id 2 prio 255 cost 10 e57) 2026-02-11T23:30:28.071+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.071+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 DeleteStart) class_id 2immediate prio 255 cost 10 e57) queued 2026-02-11T23:30:28.071+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.071+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 DeleteStart) class_id 2immediate prio 255 cost 10 e57) pg 0x46f4000 2026-02-11T23:30:28.071+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 5.0 peering, item epoch is 57, will wait on OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 DeleteStart) class_id 2immediate prio 255 cost 10 e57) 2026-02-11T23:30:28.071+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.071+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.091+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 36 ==== osd_map(57..58 src has 1..58) ==== 1200+0+0 (crc 0 0 0) 0x46eac40 con 0x396bc00 2026-02-11T23:30:28.091+0000 7fac626e8640 20 osd.0 55 OSD::ms_dispatch: osd_map(57..58 src has 1..58) 2026-02-11T23:30:28.091+0000 7fac626e8640 20 osd.0 55 _dispatch 0x46eac40 osd_map(57..58 src has 1..58) 2026-02-11T23:30:28.091+0000 7fac626e8640 10 monclient: _renew_subs 2026-02-11T23:30:28.091+0000 7fac626e8640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:28.091+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=56}) -- 0x389ad00 con 0x396bc00 2026-02-11T23:30:28.091+0000 7fac626e8640 3 osd.0 55 handle_osd_map epochs [57,58], i have 55, src has [1,58] 2026-02-11T23:30:28.091+0000 7fac626e8640 10 osd.0 55 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:28.091+0000 7fac626e8640 10 osd.0 55 handle_osd_map message skips epochs 56..56 2026-02-11T23:30:28.091+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 37 ==== osd_map(56..58 src has 1..58) ==== 1790+0+0 (crc 0 0 0) 0x46ea000 con 0x396bc00 2026-02-11T23:30:28.091+0000 7fac626e8640 20 osd.0 55 OSD::ms_dispatch: osd_map(56..58 src has 1..58) 2026-02-11T23:30:28.091+0000 7fac626e8640 20 osd.0 55 _dispatch 0x46ea000 osd_map(56..58 src has 1..58) 2026-02-11T23:30:28.091+0000 7fac626e8640 3 osd.0 55 handle_osd_map epochs [56,58], i have 55, src has [1,58] 2026-02-11T23:30:28.091+0000 7fac626e8640 10 osd.0 55 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:28.091+0000 7fac626e8640 10 osd.0 55 handle_osd_map got inc map for epoch 56 2026-02-11T23:30:28.091+0000 7fac626e8640 20 osd.0 55 got_full_map 56, nothing requested 2026-02-11T23:30:28.091+0000 7fac626e8640 10 osd.0 55 handle_osd_map got inc map for epoch 57 2026-02-11T23:30:28.091+0000 7fac626e8640 20 osd.0 55 got_full_map 57, nothing requested 2026-02-11T23:30:28.092+0000 7fac626e8640 10 osd.0 55 handle_osd_map got inc map for epoch 58 2026-02-11T23:30:28.092+0000 7fac626e8640 20 osd.0 55 got_full_map 58, nothing requested 2026-02-11T23:30:28.092+0000 7fac626e8640 20 osd.0 55 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:28.092+0000 7fac626e8640 20 osd.0 55 handle_osd_map pg_num_history pg_num_history(e58 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1}} deleted_pools ) 2026-02-11T23:30:28.092+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {56={},57={},58={}} 2026-02-11T23:30:28.092+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:28.092+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e58 maps [1~58] lci=[5,58] tlb=1) 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0 55 _committed_osd_maps 56..58 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0 55 advance to epoch 56 (<= last 58 <= newest_map 58) 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0 56 advance to epoch 57 (<= last 58 <= newest_map 58) 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0 57 advance to epoch 58 (<= last 58 <= newest_map 58) 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 58 consume_map version 58 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 55 await_reserved_maps epoch:58 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 55 await_reserved_maps done 0 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 58 consume_map 58 -- publish done 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:5.0slot:0x25809a0 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 58 identify_splits_and_merges 5.0 e55 to e58 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 58 identify_splits_and_merges 5.0 e55 to e58 pg_nums {52=1} 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 58 identify_splits_and_merges 4.0 e55 to e58 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 58 identify_splits_and_merges 4.0 e55 to e58 pg_nums {45=1} 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 58 identify_splits_and_merges 1.0 e55 to e58 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 58 identify_splits_and_merges 1.0 e55 to e58 pg_nums {24=1} 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 58 prune_pg_created 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0:0.consume_map 58 (was 55) 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 5.0 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 5.0 pending_peering first epoch 57 <= 58, requeueing 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 5.0 to_process <> waiting <> waiting_peering {57=} 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0:1.consume_map 58 (was 55) 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0:2.consume_map 58 (was 55) 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0:3.consume_map 58 (was 55) 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0:4.consume_map 58 (was 55) 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0:5.consume_map 58 (was 55) 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0:6.consume_map 58 (was 55) 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0:7.consume_map 58 (was 55) 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 DeleteStart) class_id 2immediate prio 255 cost 10 e57) queued 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 DeleteStart) class_id 2immediate prio 255 cost 10 e57) pg 0x46f4000 2026-02-11T23:30:28.092+0000 7fac5e983640 10 osd.0 58 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:28.092+0000 7fac5e983640 15 osd.0 58 enqueue_peering_evt 5.0 epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2 prio 255 cost 10 e58) 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] handle_advance_map: 56 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0:0.update_pg_epoch 5.0 55 -> 56 2026-02-11T23:30:28.092+0000 7fac5e983640 15 osd.0 58 enqueue_peering_evt 4.0 epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2 prio 255 cost 10 e58) 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] PeeringState::advance_map handle_advance_map [3,4]/[3,4] -- 3/3 2026-02-11T23:30:28.092+0000 7fac5e983640 15 osd.0 58 enqueue_peering_evt 1.0 epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:28.092+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2 prio 255 cost 10 e58) 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] state: Started advmap 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) queued 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.092+0000 7fac5e983640 7 osd.0 58 activate_map version 58 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] hit_set_clear 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] agent_stop 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.092+0000 7fac55971640 20 osd.0 58 agent_entry empty queue 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] handle_advance_map: 57 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0:0.update_pg_epoch 5.0 56 -> 57 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 56 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] PeeringState::advance_map handle_advance_map [3,4]/[3,4] -- 3/3 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] state: Started advmap 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] hit_set_clear 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] agent_stop 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] handle_advance_map: 58 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0:0.update_pg_epoch 5.0 57 -> 58 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 57 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] PeeringState::advance_map handle_advance_map [3,4]/[3,4] -- 3/3 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] state: Started advmap 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] hit_set_clear 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] agent_stop 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] handle_activate_map: epoch range: 56..58 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=15.424398422s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 55 while current is 58 2026-02-11T23:30:28.092+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] do_peering_event: epoch_sent: 57 epoch_requested: 57 DeleteStart 2026-02-11T23:30:28.092+0000 7fac55170640 5 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] exit Started/Stray 0.427140 7 0.000030 2026-02-11T23:30:28.092+0000 7fac55170640 5 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] enter Started/ToDelete 2026-02-11T23:30:28.092+0000 7fac55170640 5 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] enter Started/ToDelete/WaitDeleteReserved 2026-02-11T23:30:28.092+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 58 get_con_osd_cluster to osd.3 from_epoch 58 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 58 get_nextmap_reserved map_reservations: {58=1} 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 58 release_map epoch: 58 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 58 maybe_share_map: con v2:127.0.0.1:6828/1114810440 map epoch 55 -> 58 (shared) 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 58 send_incremental_map epoch range: (56, 58] to v2:127.0.0.1:6828/1114810440 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 58 add_map_inc_bl 56 582 bytes 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 58 add_map_inc_bl 57 574 bytes 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 58 add_map_inc_bl 58 574 bytes 2026-02-11T23:30:28.092+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- osd_map(56..58 src has 1..58) -- 0x46eae00 con 0x3741000 2026-02-11T23:30:28.092+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- pg_notify2(5.0 5.0 (query:58 sent:58 5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) ([52,54] all_participants={0, 3, 4} intervals=[([52,54] acting={0, 3, 4})])) e58/58) -- 0x486b100 con 0x3741000 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) queued 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) pg 0x4856000 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) pg 0x46f4000 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 56 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 55 -> 56 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 55 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] do_peering_event: epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:28.092+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) queued 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 NullEvt) class_id 2immediate prio 255 cost 10 e58) pg 0x40f8000 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 57 2026-02-11T23:30:28.092+0000 7fac58176640 15 osd.0 58 enqueue_peering_evt 5.0 epoch_sent: 58 epoch_requested: 58 DeleteReserved 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 56 -> 57 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 56 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:28.092+0000 7fac58176640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 DeleteReserved) class_id 2 prio 255 cost 10 e58) 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 56 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 58 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 55 -> 56 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 55 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 57 -> 58 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 57 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 56..58 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 58 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 57 2026-02-11T23:30:28.092+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 56 -> 57 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 56 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:28.092+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 DeleteReserved) class_id 2immediate prio 255 cost 10 e58) queued 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGPeeringEvent(epoch_sent: 58 epoch_requested: 58 DeleteReserved) class_id 2immediate prio 255 cost 10 e58) pg 0x46f4000 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 58 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 57 -> 58 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] do_peering_event: epoch_sent: 58 epoch_requested: 58 DeleteReserved 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 57 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:28.092+0000 7fac55170640 5 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] exit Started/ToDelete/WaitDeleteReserved 0.000172 2 0.000011 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 58 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.092+0000 7fac55170640 5 osd.0 pg_epoch: 58 pg[5.0( empty local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] enter Started/ToDelete/Deleting 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 56..58 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] on_removal 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 58 2026-02-11T23:30:28.092+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] on_shutdown 2026-02-11T23:30:28.092+0000 7fac51168640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 58 epoch_requested: 58 NullEvt 2026-02-11T23:30:28.092+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:28.092+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:28.092+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] clear_backoffs 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] exit NotTrimming 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] enter NotTrimming 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] on_change 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] clear_async_reads 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] clear_primary_state 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] release_backoffs [5:00000000::::0,MAX) 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:28.092+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:28.092+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] agent_stop 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] cancel_recovery 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] clear_recovery_state 2026-02-11T23:30:28.092+0000 7fac55170640 15 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:28.092+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 58 queue_for_pg_delete on 5.0 e 58 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(5.0 PGDelete(pgid=5.0 epoch_queued=58) class_id 1 prio 5 cost 1048576 e58) 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGDelete(5.0 e58) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e58) queued 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process 5.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(5.0 PGDelete(5.0 e58) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e58) pg 0x46f4000 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] do_peering_event: epoch_sent: 58 epoch_requested: 58 DeleteSome 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] do_delete_work 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] do_delete_work [#5:00000000::::head#] 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 58 try_finish_pg_delete 5.0 0x46f4000 2026-02-11T23:30:28.092+0000 7fac55170640 10 osd.0:0._detach_pg 5.0 0x46f4000 2026-02-11T23:30:28.092+0000 7fac55170640 5 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 DELETING pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] exit Started/ToDelete/Deleting 0.000202 1 0.000035 2026-02-11T23:30:28.092+0000 7fac55170640 5 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] exit Started/ToDelete 0.000380 0 0.000000 2026-02-11T23:30:28.092+0000 7fac55170640 5 osd.0 pg_epoch: 58 pg[5.0( empty lb MIN local-lis/les=52/53 n=0 ec=52/52 lis/c=52/52 les/c/f=53/53/0 sis=55 pruub=14.997283936s) [3,4] r=-1 lpr=55 pi=[52,55)/1 crt=0'0 unknown NOTIFY pruub 35.925540924s@ mbc={}] exit Started 0.427531 0 0.000000 2026-02-11T23:30:28.092+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.092+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.092+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:28.092+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:28.092+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:28.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:28.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:58.168503+0000) 2026-02-11T23:30:28.216+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 17 ==== pg_lease(1.0 pg_lease(ru 27.071365356s ub 35.071456909s int 16.000000000s) e57/57) ==== 58+0+0 (unknown 0 0 0) 0x3b1a300 con 0x3752000 2026-02-11T23:30:28.216+0000 7fac703fa640 15 osd.0 58 enqueue_peering_evt 1.0 epoch_sent: 57 epoch_requested: 57 MLease epoch 57 from osd.1 pg_lease(ru 27.071365356s ub 35.071456909s int 16.000000000s) 2026-02-11T23:30:28.216+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MLease epoch 57 from osd.1 pg_lease(ru 27.071365356s ub 35.071456909s int 16.000000000s)) class_id 2 prio 255 cost 10 e57) 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MLease epoch 57 from osd.1 pg_lease(ru 27.071365356s ub 35.071456909s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e57) queued 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 57 epoch_requested: 57 MLease epoch 57 from osd.1 pg_lease(ru 27.071365356s ub 35.071456909s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e57) pg 0x40f8000 2026-02-11T23:30:28.216+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.216+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 57 epoch_requested: 57 MLease epoch 57 from osd.1 pg_lease(ru 27.071365356s ub 35.071456909s int 16.000000000s) 2026-02-11T23:30:28.216+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 27.071365356s ub 35.071456909s int 16.000000000s) 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -1.980759859s -> ru 29.052125931s 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 29.052125931s 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.980949879s -> ruub 37.052406311s 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 37.052406311s 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 58 get_con_osd_cluster to osd.1 from_epoch 58 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 58 get_nextmap_reserved map_reservations: {58=1} 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 58 release_map epoch: 58 2026-02-11T23:30:28.216+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_lease_ack(1.0 pg_lease_ack(ruub 35.071456909s) e58/58) -- 0x2580160 con 0x3752000 2026-02-11T23:30:28.216+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.216+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.287+0000 7fac683ea640 10 osd.0 58 tick_without_osd_lock 2026-02-11T23:30:28.287+0000 7fac683ea640 0 osd.0 58 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:28.287+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:28.287+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:28.287+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:28.287+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.152 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:28.287+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:28.288, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:28.287+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:28.287+0000 7fac683ea640 10 osd.0 58 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:28.287+0000 7fac683ea640 20 osd.0 58 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:28.287+0000 7fac683ea640 10 osd.0 58 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:28.287+0000 7fac683ea640 20 osd.0 58 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:28.348+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 6 ==== osd_ping(ping e59 up_from 9 ping_stamp 2026-02-11T23:30:28.349831+0000/19.203395844s send_stamp 19.203395844s delta_ub -1.980622172s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b980 con 0x396a400 2026-02-11T23:30:28.348+0000 7fac71bfd640 20 osd.0 58 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980921984s,-1.980622172s]) 2026-02-11T23:30:28.348+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e58 up_from 5 ping_stamp 2026-02-11T23:30:28.349831+0000/19.203395844s send_stamp 21.184316635s delta_ub 1.980921984s) -- 0x3b1a600 con 0x396a400 2026-02-11T23:30:28.348+0000 7fac71bfd640 20 osd.0 58 get_con_osd_cluster to osd.1 from_epoch 58 2026-02-11T23:30:28.348+0000 7fac71bfd640 20 osd.0 58 get_nextmap_reserved map_reservations: {58=1} 2026-02-11T23:30:28.348+0000 7fac71bfd640 20 osd.0 58 release_map epoch: 58 2026-02-11T23:30:28.348+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 6 ==== osd_ping(ping e59 up_from 9 ping_stamp 2026-02-11T23:30:28.349831+0000/19.203395844s send_stamp 19.203395844s delta_ub -1.980622172s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bb00 con 0x396a000 2026-02-11T23:30:28.348+0000 7fac71bfd640 10 osd.0 58 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 47 to ping map epoch of 59 2026-02-11T23:30:28.348+0000 7fac71bfd640 10 osd.0 58 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 58 is not newer than session's projected_epoch of 59 2026-02-11T23:30:28.348+0000 7fac723fe640 20 osd.0 58 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980953693s,-1.980622172s]) 2026-02-11T23:30:28.348+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e58 up_from 5 ping_stamp 2026-02-11T23:30:28.349831+0000/19.203395844s send_stamp 21.184350967s delta_ub 1.980953693s) -- 0x3b1be00 con 0x396a000 2026-02-11T23:30:28.348+0000 7fac723fe640 20 osd.0 58 get_con_osd_cluster to osd.1 from_epoch 58 2026-02-11T23:30:28.348+0000 7fac723fe640 20 osd.0 58 get_nextmap_reserved map_reservations: {58=1} 2026-02-11T23:30:28.348+0000 7fac723fe640 20 osd.0 58 release_map epoch: 58 2026-02-11T23:30:28.348+0000 7fac723fe640 10 osd.0 58 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 58 is not newer than session's projected_epoch of 59 2026-02-11T23:30:28.349+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 18 ==== osd_map(59..59 src has 1..59) ==== 635+0+0 (unknown 0 0 0) 0x3940540 con 0x3752000 2026-02-11T23:30:28.349+0000 7fac606e4640 20 osd.0 58 OSD::ms_dispatch: osd_map(59..59 src has 1..59) 2026-02-11T23:30:28.349+0000 7fac606e4640 20 osd.0 58 _dispatch 0x3940540 osd_map(59..59 src has 1..59) 2026-02-11T23:30:28.349+0000 7fac606e4640 3 osd.0 58 handle_osd_map epochs [59,59], i have 58, src has [1,59] 2026-02-11T23:30:28.349+0000 7fac606e4640 10 osd.0 58 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:28.349+0000 7fac606e4640 10 osd.0 58 handle_osd_map got inc map for epoch 59 2026-02-11T23:30:28.349+0000 7fac606e4640 20 osd.0 58 got_full_map 59, nothing requested 2026-02-11T23:30:28.349+0000 7fac606e4640 10 osd.0 58 _track_pools_and_pg_num_changes recording new pool 6 pg_num 1 2026-02-11T23:30:28.349+0000 7fac606e4640 20 osd.0 58 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:28.349+0000 7fac606e4640 20 osd.0 58 handle_osd_map pg_num_history pg_num_history(e59 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1}} deleted_pools ) 2026-02-11T23:30:28.349+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {59={}} 2026-02-11T23:30:28.349+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:28.349+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e59 maps [1~59] lci=[5,59] tlb=1) 2026-02-11T23:30:28.349+0000 7fac5e983640 10 osd.0 58 _committed_osd_maps 59..59 2026-02-11T23:30:28.349+0000 7fac5e983640 10 osd.0 58 advance to epoch 59 (<= last 59 <= newest_map 59) 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0 59 consume_map version 59 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0 58 await_reserved_maps epoch:59 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0 58 await_reserved_maps done 0 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0 59 consume_map 59 -- publish done 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:5.0slot:0x25809a0 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 5.0 has no pg and waiting_for_split 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0 59 identify_splits_and_merges 4.0 e58 to e59 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0 59 identify_splits_and_merges 4.0 e58 to e59 pg_nums {45=1} 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0 59 identify_splits_and_merges 1.0 e58 to e59 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0 59 identify_splits_and_merges 1.0 e58 to e59 pg_nums {24=1} 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0 59 prune_pg_created 2026-02-11T23:30:28.349+0000 7fac5e983640 10 osd.0:0.consume_map 59 (was 58) 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 5.0 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 5.0 empty, pruning 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:28.349+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:28.349+0000 7fac5e983640 10 osd.0:1.consume_map 59 (was 58) 2026-02-11T23:30:28.349+0000 7fac5e983640 10 osd.0:2.consume_map 59 (was 58) 2026-02-11T23:30:28.349+0000 7fac5e983640 10 osd.0:3.consume_map 59 (was 58) 2026-02-11T23:30:28.349+0000 7fac5e983640 10 osd.0:4.consume_map 59 (was 58) 2026-02-11T23:30:28.349+0000 7fac5e983640 10 osd.0:5.consume_map 59 (was 58) 2026-02-11T23:30:28.349+0000 7fac5e983640 10 osd.0:6.consume_map 59 (was 58) 2026-02-11T23:30:28.350+0000 7fac5e983640 10 osd.0:7.consume_map 59 (was 58) 2026-02-11T23:30:28.350+0000 7fac5e983640 10 osd.0 59 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:28.350+0000 7fac5e983640 15 osd.0 59 enqueue_peering_evt 4.0 epoch_sent: 59 epoch_requested: 59 NullEvt 2026-02-11T23:30:28.350+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt) class_id 2 prio 255 cost 10 e59) 2026-02-11T23:30:28.350+0000 7fac5e983640 15 osd.0 59 enqueue_peering_evt 1.0 epoch_sent: 59 epoch_requested: 59 NullEvt 2026-02-11T23:30:28.350+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt) class_id 2 prio 255 cost 10 e59) 2026-02-11T23:30:28.350+0000 7fac5e983640 7 osd.0 59 activate_map version 59 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt) class_id 2immediate prio 255 cost 10 e59) queued 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt) class_id 2immediate prio 255 cost 10 e59) pg 0x4856000 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 59 2026-02-11T23:30:28.350+0000 7fac55971640 20 osd.0 59 agent_entry empty queue 2026-02-11T23:30:28.350+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.350+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt) class_id 2immediate prio 255 cost 10 e59) queued 2026-02-11T23:30:28.350+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.350+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt) class_id 2immediate prio 255 cost 10 e59) pg 0x40f8000 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 58 -> 59 2026-02-11T23:30:28.350+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 59 2026-02-11T23:30:28.350+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 58 -> 59 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 pg_epoch: 58 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:28.350+0000 7fac55170640 10 osd.0 pg_epoch: 58 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:28.350+0000 7fac55170640 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:28.350+0000 7fac55170640 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 59 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 59..59 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:28.350+0000 7fac55170640 20 osd.0 59 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 59 2026-02-11T23:30:28.350+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 59 epoch_requested: 59 NullEvt 2026-02-11T23:30:28.350+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:28.350+0000 7fac55170640 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 59..59 2026-02-11T23:30:28.350+0000 7fac55170640 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.350+0000 7fac55170640 20 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 59 2026-02-11T23:30:28.350+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.350+0000 7fac55170640 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 59 epoch_requested: 59 NullEvt 2026-02-11T23:30:28.350+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.350+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.350+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 18 ==== pg_query2(6.0 6.0 query(info 0'0 epoch_sent 59) e59/59) ==== 158+0+0 (unknown 0 0 0) 0x3f3f680 con 0x3e4a000 2026-02-11T23:30:28.350+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 17 ==== osd_map(50..59 src has 1..59) ==== 5576+0+0 (unknown 0 0 0) 0x3940fc0 con 0x3e4a000 2026-02-11T23:30:28.350+0000 7fac606e4640 20 osd.0 59 OSD::ms_dispatch: osd_map(50..59 src has 1..59) 2026-02-11T23:30:28.350+0000 7fac703fa640 15 osd.0 59 enqueue_peering_evt 6.0 epoch_sent: 59 epoch_requested: 59 MQuery 6.0 from 4 query_epoch 59 query: query(info 0'0 epoch_sent 59) 2026-02-11T23:30:28.350+0000 7fac606e4640 20 osd.0 59 _dispatch 0x3940fc0 osd_map(50..59 src has 1..59) 2026-02-11T23:30:28.350+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 MQuery 6.0 from 4 query_epoch 59 query: query(info 0'0 epoch_sent 59)) class_id 2 prio 255 cost 10 e59) 2026-02-11T23:30:28.350+0000 7fac606e4640 3 osd.0 59 handle_osd_map epochs [50,59], i have 59, src has [1,59] 2026-02-11T23:30:28.350+0000 7fac606e4640 10 osd.0 59 handle_osd_map: received 0 full maps and 10 incremental maps 2026-02-11T23:30:28.350+0000 7fac606e4640 10 osd.0 59 no new maps here, dropping 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 MQuery 6.0 from 4 query_epoch 59 query: query(info 0'0 epoch_sent 59)) class_id 2immediate prio 255 cost 10 e59) queued 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 MQuery 6.0 from 4 query_epoch 59 query: query(info 0'0 epoch_sent 59)) class_id 2immediate prio 255 cost 10 e59) pg 0 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 59 handle_pg_query_nopg 6.0 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 59 pg 6.0 dne 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 59 get_con_osd_cluster to osd.4 from_epoch 59 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 59 get_nextmap_reserved map_reservations: {59=1} 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 59 release_map epoch: 59 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 49 -> 59 (shared) 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 59 send_incremental_map epoch range: (50, 59] to v2:127.0.0.1:6836/864727417 2026-02-11T23:30:28.350+0000 7fac51168640 10 osd.0 59 add_map_inc_bl 59 591 bytes 2026-02-11T23:30:28.350+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(50..59 src has 1..59) -- 0x3941880 con 0x3e4a000 2026-02-11T23:30:28.350+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(6.0 6.0 (query:59 sent:59 6.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e59/59) -- 0x4868700 con 0x3e4a000 2026-02-11T23:30:28.350+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.350+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.396+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 19 ==== PGlog(6.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.345008850s int 16.000000000s) e60/59) ==== 1172+0+0 (unknown 0 0 0) 0x3901800 con 0x3e4a000 2026-02-11T23:30:28.396+0000 7fac703fa640 15 osd.0 59 enqueue_peering_evt 6.0 epoch_sent: 60 epoch_requested: 59 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.345008850s int 16.000000000s) +create_info 2026-02-11T23:30:28.396+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 59 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.345008850s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e60) 2026-02-11T23:30:28.396+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.396+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 59 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.345008850s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:28.396+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.396+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 59 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.345008850s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e60) pg 0 2026-02-11T23:30:28.396+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 map 60 > 59 2026-02-11T23:30:28.396+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 6.0 peering, item epoch is 60, will wait on OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 59 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.345008850s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e60) 2026-02-11T23:30:28.396+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.396+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.668+0000 7fac4d160640 5 osd.0 59 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x3578a/0x84000, compress 0x0/0x0/0x0, omap 0x6470, meta 0x1a49b90), peers [1,2,3,4] op hist []) 2026-02-11T23:30:28.668+0000 7fac4d160640 20 osd.0 59 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:28.668+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e59 up_from 5 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 21.504076004s delta_ub 1.980953693s) -- 0x3b1b680 con 0x396b400 2026-02-11T23:30:28.668+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e59 up_from 5 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 21.504076004s delta_ub 1.980953693s) -- 0x3b1af00 con 0x3752400 2026-02-11T23:30:28.668+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e59 up_from 5 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 21.504076004s delta_ub 4.649135590s) -- 0x3b1a480 con 0x396b800 2026-02-11T23:30:28.668+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e59 up_from 5 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 21.504076004s delta_ub 4.649135590s) -- 0x3b1ac00 con 0x34c1000 2026-02-11T23:30:28.668+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e59 up_from 5 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 21.504076004s delta_ub 7.310479641s) -- 0x3b1a780 con 0x3740000 2026-02-11T23:30:28.668+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e59 up_from 5 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 21.504076004s delta_ub 7.310479641s) -- 0x3b1b380 con 0x3740400 2026-02-11T23:30:28.668+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e59 up_from 5 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 21.504076004s delta_ub 9.887022972s) -- 0x3b1a900 con 0x3740800 2026-02-11T23:30:28.668+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e59 up_from 5 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 21.504076004s delta_ub 9.887022972s) -- 0x3b1bc80 con 0x3740c00 2026-02-11T23:30:28.668+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 6 ==== osd_ping(ping_reply e59 up_from 9 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 19.523330688s delta_ub -1.980746627s) ==== 2033+0+0 (crc 0 0 0) 0x3b1aa80 con 0x396b400 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 get_con_osd_cluster to osd.1 from_epoch 59 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 get_nextmap_reserved map_reservations: {59=1} 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 release_map epoch: 59 2026-02-11T23:30:28.668+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 6 ==== osd_ping(ping_reply e59 up_from 9 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 19.523376465s delta_ub -1.980700016s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ac00 con 0x3752400 2026-02-11T23:30:28.668+0000 7fac703fa640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 59 is not newer than session's projected_epoch of 59 2026-02-11T23:30:28.668+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 5 ==== osd_ping(ping_reply e59 up_from 13 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 16.855184555s delta_ub -4.648892403s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a900 con 0x396b800 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980938911s,-1.980746627s]) 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 get_con_osd_cluster to osd.2 from_epoch 59 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 get_nextmap_reserved map_reservations: {59=1} 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 release_map epoch: 59 2026-02-11T23:30:28.668+0000 7fac723fe640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 41 to ping map epoch of 59 2026-02-11T23:30:28.668+0000 7fac723fe640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 59 is not newer than session's projected_epoch of 59 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649097919s,-4.648892403s]) 2026-02-11T23:30:28.668+0000 7fac71bfd640 20 osd.0 59 get_con_osd_cluster to osd.1 from_epoch 59 2026-02-11T23:30:28.668+0000 7fac71bfd640 20 osd.0 59 get_nextmap_reserved map_reservations: {59=1} 2026-02-11T23:30:28.668+0000 7fac71bfd640 20 osd.0 59 release_map epoch: 59 2026-02-11T23:30:28.668+0000 7fac71bfd640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 59 is not newer than session's projected_epoch of 59 2026-02-11T23:30:28.668+0000 7fac71bfd640 20 osd.0 59 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980925322s,-1.980700016s]) 2026-02-11T23:30:28.668+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 5 ==== osd_ping(ping_reply e59 up_from 13 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 16.855215073s delta_ub -4.648861408s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ac00 con 0x34c1000 2026-02-11T23:30:28.668+0000 7fac71bfd640 20 osd.0 59 get_con_osd_cluster to osd.2 from_epoch 59 2026-02-11T23:30:28.668+0000 7fac71bfd640 20 osd.0 59 get_nextmap_reserved map_reservations: {59=1} 2026-02-11T23:30:28.668+0000 7fac71bfd640 20 osd.0 59 release_map epoch: 59 2026-02-11T23:30:28.668+0000 7fac71bfd640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 59 is not newer than session's projected_epoch of 59 2026-02-11T23:30:28.668+0000 7fac71bfd640 20 osd.0 59 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649125099s,-4.648861408s]) 2026-02-11T23:30:28.668+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 3 ==== osd_ping(ping_reply e60 up_from 17 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 14.193858147s delta_ub -7.310217857s) ==== 2033+0+0 (crc 0 0 0) 0x3b1aa80 con 0x3740000 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 get_con_osd_cluster to osd.3 from_epoch 59 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 get_nextmap_reserved map_reservations: {59=1} 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 release_map epoch: 59 2026-02-11T23:30:28.668+0000 7fac703fa640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 58 to ping map epoch of 60 2026-02-11T23:30:28.668+0000 7fac703fa640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 59 is not newer than session's projected_epoch of 60 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310616016s,-7.310217857s]) 2026-02-11T23:30:28.668+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 20 ==== osd_map(60..60 src has 1..60) ==== 296+0+0 (unknown 0 0 0) 0x46eae00 con 0x3741000 2026-02-11T23:30:28.668+0000 7fac606e4640 20 osd.0 59 OSD::ms_dispatch: osd_map(60..60 src has 1..60) 2026-02-11T23:30:28.668+0000 7fac606e4640 20 osd.0 59 _dispatch 0x46eae00 osd_map(60..60 src has 1..60) 2026-02-11T23:30:28.668+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 3 ==== osd_ping(ping_reply e60 up_from 17 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 14.194066048s delta_ub -7.310010433s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x3740400 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 get_con_osd_cluster to osd.3 from_epoch 59 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 get_nextmap_reserved map_reservations: {59=1} 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 release_map epoch: 59 2026-02-11T23:30:28.668+0000 7fac723fe640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 59 is not newer than session's projected_epoch of 60 2026-02-11T23:30:28.668+0000 7fac606e4640 3 osd.0 59 handle_osd_map epochs [60,60], i have 59, src has [1,60] 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310455799s,-7.310010433s]) 2026-02-11T23:30:28.668+0000 7fac606e4640 10 osd.0 59 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:28.668+0000 7fac606e4640 10 osd.0 59 handle_osd_map got inc map for epoch 60 2026-02-11T23:30:28.668+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 3 ==== osd_ping(ping_reply e60 up_from 21 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 11.617576599s delta_ub -9.886499405s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a300 con 0x3740c00 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 get_con_osd_cluster to osd.4 from_epoch 59 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 get_nextmap_reserved map_reservations: {59=1} 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 release_map epoch: 59 2026-02-11T23:30:28.668+0000 7fac703fa640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 59 to ping map epoch of 60 2026-02-11T23:30:28.668+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 3 ==== osd_ping(ping_reply e60 up_from 21 ping_stamp 2026-02-11T23:30:28.669661+0000/21.504076004s send_stamp 11.617362022s delta_ub -9.886713982s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x3740800 2026-02-11T23:30:28.668+0000 7fac703fa640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 59 is not newer than session's projected_epoch of 60 2026-02-11T23:30:28.668+0000 7fac703fa640 20 osd.0 59 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886967659s,-9.886499405s]) 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 get_con_osd_cluster to osd.4 from_epoch 59 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 get_nextmap_reserved map_reservations: {59=1} 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 release_map epoch: 59 2026-02-11T23:30:28.668+0000 7fac723fe640 10 osd.0 59 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 59 is not newer than session's projected_epoch of 60 2026-02-11T23:30:28.668+0000 7fac723fe640 20 osd.0 59 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887206078s,-9.886713982s]) 2026-02-11T23:30:28.668+0000 7fac606e4640 20 osd.0 59 got_full_map 60, nothing requested 2026-02-11T23:30:28.668+0000 7fac606e4640 20 osd.0 59 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:28.668+0000 7fac606e4640 20 osd.0 59 handle_osd_map pg_num_history pg_num_history(e60 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1}} deleted_pools ) 2026-02-11T23:30:28.668+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {60={}} 2026-02-11T23:30:28.668+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:28.668+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e60 maps [1~60] lci=[5,60] tlb=1) 2026-02-11T23:30:28.668+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 20 ==== osd_map(60..60 src has 1..60) ==== 296+0+0 (unknown 0 0 0) 0x3941880 con 0x3e4a000 2026-02-11T23:30:28.668+0000 7fac606e4640 20 osd.0 59 OSD::ms_dispatch: osd_map(60..60 src has 1..60) 2026-02-11T23:30:28.668+0000 7fac606e4640 20 osd.0 59 _dispatch 0x3941880 osd_map(60..60 src has 1..60) 2026-02-11T23:30:28.668+0000 7fac606e4640 3 osd.0 59 handle_osd_map epochs [60,60], i have 60, src has [1,60] 2026-02-11T23:30:28.668+0000 7fac606e4640 10 osd.0 59 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:28.668+0000 7fac606e4640 10 osd.0 59 no new maps here, dropping 2026-02-11T23:30:28.669+0000 7fac5e983640 10 osd.0 59 _committed_osd_maps 60..60 2026-02-11T23:30:28.669+0000 7fac5e983640 10 osd.0 59 advance to epoch 60 (<= last 60 <= newest_map 60) 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0 60 consume_map version 60 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0 59 await_reserved_maps epoch:60 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0 59 await_reserved_maps done 0 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0 60 consume_map 60 -- publish done 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:6.0slot:0x25d91e0 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 6.0 has no pg and waiting_for_split 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0 60 identify_splits_and_merges 4.0 e59 to e60 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0 60 identify_splits_and_merges 4.0 e59 to e60 pg_nums {45=1} 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0 60 identify_splits_and_merges 1.0 e59 to e60 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0 60 identify_splits_and_merges 1.0 e59 to e60 pg_nums {24=1} 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0 60 prune_pg_created 2026-02-11T23:30:28.669+0000 7fac5e983640 10 osd.0:0.consume_map 60 (was 59) 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 6.0 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 6.0 pending_peering first epoch 60 <= 60, requeueing 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 6.0 to_process <> waiting <> waiting_peering {60=} 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:28.669+0000 7fac5e983640 10 osd.0:1.consume_map 60 (was 59) 2026-02-11T23:30:28.669+0000 7fac5e983640 10 osd.0:2.consume_map 60 (was 59) 2026-02-11T23:30:28.669+0000 7fac5e983640 10 osd.0:3.consume_map 60 (was 59) 2026-02-11T23:30:28.669+0000 7fac5e983640 10 osd.0:4.consume_map 60 (was 59) 2026-02-11T23:30:28.669+0000 7fac5e983640 10 osd.0:5.consume_map 60 (was 59) 2026-02-11T23:30:28.669+0000 7fac5e983640 10 osd.0:6.consume_map 60 (was 59) 2026-02-11T23:30:28.669+0000 7fac5e983640 10 osd.0:7.consume_map 60 (was 59) 2026-02-11T23:30:28.669+0000 7fac5e983640 10 osd.0 60 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:28.669+0000 7fac5e983640 15 osd.0 60 enqueue_peering_evt 4.0 epoch_sent: 60 epoch_requested: 60 NullEvt 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 59 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.345008850s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2 prio 255 cost 10 e60) 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 59 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.345008850s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e60) pg 0 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 no pg, should create on OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 59 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.345008850s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e60) 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 60 _make_pg 6.0 2026-02-11T23:30:28.669+0000 7fac51168640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:6, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:30:28.669+0000 7fac51168640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:6, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:30:28.669+0000 7fac51168640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:6, shard:-1, shard_prefix: , prefixes: {"0000000000000006."}] update_bits prefix updated 2026-02-11T23:30:28.669+0000 7fac51168640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:6, shard:-1, shard_prefix: , prefixes: {"0000000000000006."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:30:28.669+0000 7fac51168640 5 osd.0 pg_epoch: 59 pg[6.0(unlocked)] enter Initial 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 pg_epoch: 59 pg[6.0(unlocked)] enter NotTrimming 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role 2 up [4,2,0] acting [4,2,0] history ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59 past_intervals ([0,0] all_participants={} intervals=[]) 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4,2,0]+[4,2,0] 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [0x3863810] 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 21.505155563s + 0.000000000s) 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:30:28.669+0000 7fac51168640 15 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:28.669+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:28.669+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:28.669+0000 7fac51168640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:30:28.669+0000 7fac51168640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:30:28.669+0000 7fac51168640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:30:28.669+0000 7fac51168640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:30:28.669+0000 7fac51168640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:30:28.669+0000 7fac51168640 10 end of _write_log_and_missing 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:30:28.669+0000 7fac51168640 5 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] exit Initial 0.000097 0 0.000000 2026-02-11T23:30:28.669+0000 7fac51168640 5 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=0 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 59 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 59..59 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:28.669+0000 7fac51168640 5 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] exit Reset 0.000018 1 0.000016 2026-02-11T23:30:28.669+0000 7fac51168640 5 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:30:28.669+0000 7fac51168640 5 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:30:28.669+0000 7fac51168640 1 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:30:28.669+0000 7fac51168640 5 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] exit Start 0.000003 0 0.000000 2026-02-11T23:30:28.669+0000 7fac51168640 5 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 59 while current is 59 2026-02-11T23:30:28.669+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 60 handle_pg_create_info new pg pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0:0._attach_pg 6.0 0x48c8000 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0:0._wake_pg_slot _wake_pg_slot 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 60 identify_splits_and_merges 6.0 e59 to e60 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 60 identify_splits_and_merges 6.0 e59 to e60 pg_nums {59=1} 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0:0._prime_splits 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] handle_advance_map: 60 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0:0.update_pg_epoch 6.0 59 -> 60 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 59 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [4,2,0]/[4,2,0] -- 4/4 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:28.669+0000 7fac5e983640 15 osd.0 60 enqueue_peering_evt 1.0 epoch_sent: 60 epoch_requested: 60 NullEvt 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.669+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2 prio 255 cost 10 e60) 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2immediate prio 255 cost 10 e60) pg 0x4856000 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 60 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.669+0000 7fac5e983640 7 osd.0 60 activate_map version 60 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 60..60 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 60 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 59 -> 60 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 59 while current is 60 2026-02-11T23:30:28.669+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 60 epoch_requested: 59 MLogRec from 4 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 27.345008850s int 16.000000000s) +create_info 2026-02-11T23:30:28.669+0000 7fac55971640 20 osd.0 60 agent_entry empty queue 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] state: got info+log from osd.4 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) log((0'0,0'0], crt=0'0) 2026-02-11T23:30:28.669+0000 7fac51168640 10 merge_log log((0'0,0'0], crt=0'0) from osd.4 into log((0'0,0'0], crt=0'0) 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 59 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:28.669+0000 7fac51168640 5 merge_log_dups log.dups.size()=0olog.dups.size()=0 2026-02-11T23:30:28.669+0000 7fac51168640 5 end of merge_log_dups changed=0 log.dups.size()=0 olog.dups.size()=0 2026-02-11T23:30:28.669+0000 7fac51168640 10 merge_log result log((0'0,0'0], crt=0'0) missing(0 may_include_deletes = 1) changed=0 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 27.345008850s int 16.000000000s) 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.887206078s -> ruub 37.232215881s 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] PeeringState::proc_lease readable_until_ub now 37.232215881s 2026-02-11T23:30:28.669+0000 7fac51168640 5 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] exit Started/Stray 0.000128 3 0.000025 2026-02-11T23:30:28.669+0000 7fac51168640 5 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.669+0000 7fac51168640 5 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=0/0 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 60 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 0'0 -> 0'0 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 60..60 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:30:28.669+0000 7fac51168640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 60 2026-02-11T23:30:28.669+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 60 epoch_requested: 60 NullEvt 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:30:28.669+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.669+0000 7fac51168640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.669+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:30:28.669+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 NullEvt) class_id 2immediate prio 255 cost 10 e60) pg 0x40f8000 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 60 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 59 -> 60 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 60 advance_pg new pool opts old pool opts 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 60..60 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 60 2026-02-11T23:30:28.669+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 60 epoch_requested: 60 NullEvt 2026-02-11T23:30:28.669+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.669+0000 7fac55170640 15 osd.0 60 enqueue_peering_evt 6.0 epoch_sent: 60 epoch_requested: 60 ActivateCommitted from 60 processed at 60 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 ActivateCommitted from 60 processed at 60) class_id 2 prio 255 cost 10 e60) 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 ActivateCommitted from 60 processed at 60) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 ActivateCommitted from 60 processed at 60) class_id 2immediate prio 255 cost 10 e60) pg 0x48c8000 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 60 epoch_requested: 60 ActivateCommitted from 60 processed at 60 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 unknown mbc={}] state: react 60 telling primary 2026-02-11T23:30:28.669+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.669+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 60 get_con_osd_cluster to osd.4 from_epoch 60 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 60 get_nextmap_reserved map_reservations: {60=1} 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 60 release_map epoch: 60 2026-02-11T23:30:28.669+0000 7fac55170640 10 osd.0 60 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 60 is not newer than session's projected_epoch of 60 2026-02-11T23:30:28.669+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_info2(6.0 6.0 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/0 les/c/f=60/0/0 sis=59) e60/60) -- 0x486b800 con 0x3e4a000 2026-02-11T23:30:28.669+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.723+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 21 ==== pg_lease(6.0 pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s) e60/60) ==== 58+0+0 (unknown 0 0 0) 0x3b1a300 con 0x3e4a000 2026-02-11T23:30:28.723+0000 7fac703fa640 15 osd.0 60 enqueue_peering_evt 6.0 epoch_sent: 60 epoch_requested: 60 MLease epoch 60 from osd.4 pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s) 2026-02-11T23:30:28.723+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MLease epoch 60 from osd.4 pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s)) class_id 2 prio 255 cost 10 e60) 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MLease epoch 60 from osd.4 pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MLease epoch 60 from osd.4 pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e60) pg 0x48c8000 2026-02-11T23:30:28.723+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 60 epoch_requested: 60 MLease epoch 60 from osd.4 pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s) 2026-02-11T23:30:28.723+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 22 ==== pg_info2(6.0 6.0 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/0 les/c/f=60/0/0 sis=59) e60/60) ==== 1126+0+0 (unknown 0 0 0) 0x486b800 con 0x3e4a000 2026-02-11T23:30:28.723+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.723+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s) 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.886713982s -> ru 37.231723785s 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 37.231723785s 2026-02-11T23:30:28.723+0000 7fac703fa640 15 osd.0 60 enqueue_peering_evt 6.0 epoch_sent: 60 epoch_requested: 60 MInfoRec from 4 info: 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/0 les/c/f=60/0/0 sis=59) pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s) 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.887206078s -> ruub 37.559883118s 2026-02-11T23:30:28.723+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MInfoRec from 4 info: 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/0 les/c/f=60/0/0 sis=59) pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s)) class_id 2 prio 255 cost 10 e60) 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 37.559883118s 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 60 get_con_osd_cluster to osd.4 from_epoch 60 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 60 get_nextmap_reserved map_reservations: {60=1} 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 60 release_map epoch: 60 2026-02-11T23:30:28.723+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.723+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_lease_ack(6.0 pg_lease_ack(ruub 27.672676086s) e60/60) -- 0x25d9760 con 0x3e4a000 2026-02-11T23:30:28.723+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MInfoRec from 4 info: 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/0 les/c/f=60/0/0 sis=59) pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:28.723+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:28.723+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.723+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.723+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MInfoRec from 4 info: 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/0 les/c/f=60/0/0 sis=59) pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e60) pg 0x48c8000 2026-02-11T23:30:28.723+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=0/0 les/c/f=0/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 60 epoch_requested: 60 MInfoRec from 4 info: 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/0 les/c/f=60/0/0 sis=59) pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s) 2026-02-11T23:30:28.723+0000 7fac51168640 20 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/0 les/c/f=60/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=59/59 lis/c=59/0 les/c/f=60/0/0 sis=59 2026-02-11T23:30:28.723+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:28.723+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.723+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 23 ==== pg_info2(6.0 6.0 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) e60/60) ==== 1126+0+0 (unknown 0 0 0) 0x486b800 con 0x3e4a000 2026-02-11T23:30:28.723+0000 7fac703fa640 15 osd.0 60 enqueue_peering_evt 6.0 epoch_sent: 60 epoch_requested: 60 MInfoRec from 4 info: 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s) 2026-02-11T23:30:28.723+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MInfoRec from 4 info: 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s)) class_id 2 prio 255 cost 10 e60) 2026-02-11T23:30:28.723+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:28.724+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MInfoRec from 4 info: 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e60) queued 2026-02-11T23:30:28.724+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:28.724+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 60 epoch_requested: 60 MInfoRec from 4 info: 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e60) pg 0x48c8000 2026-02-11T23:30:28.724+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/0 les/c/f=60/0/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 60 epoch_requested: 60 MInfoRec from 4 info: 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) pg_lease(ru 27.345008850s ub 27.672676086s int 16.000000000s) 2026-02-11T23:30:28.724+0000 7fac51168640 20 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::update_history advanced history from ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59 2026-02-11T23:30:28.724+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.724+0000 7fac51168640 20 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:30:28.724+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:28.724+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.724+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.724+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.724+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:28.724+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.041+0000 7fac68beb640 10 osd.0 60 tick 2026-02-11T23:30:29.041+0000 7fac68beb640 20 osd.0 60 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:29.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:29.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:29.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:29:59.168527+0000) 2026-02-11T23:30:29.297+0000 7fac683ea640 10 osd.0 60 tick_without_osd_lock 2026-02-11T23:30:29.297+0000 7fac683ea640 0 osd.0 60 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:29.297+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:29.297+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.152 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:29.297+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:29.298, recovery is active?:false restrictions:<....> 2026-02-11T23:30:29.297+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:29.297+0000 7fac683ea640 10 osd.0 60 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:29.297+0000 7fac683ea640 20 osd.0 60 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:29.297+0000 7fac683ea640 10 osd.0 60 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:29.297+0000 7fac683ea640 20 osd.0 60 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:29.448+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 7 ==== osd_ping(ping e59 up_from 9 ping_stamp 2026-02-11T23:30:29.450036+0000/20.303600311s send_stamp 20.303600311s delta_ub -1.980700016s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ac00 con 0x396a400 2026-02-11T23:30:29.448+0000 7fac71bfd640 20 osd.0 60 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980907917s,-1.980700016s]) 2026-02-11T23:30:29.448+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e60 up_from 5 ping_stamp 2026-02-11T23:30:29.450036+0000/20.303600311s send_stamp 22.284509659s delta_ub 1.980907917s) -- 0x3b1af00 con 0x396a400 2026-02-11T23:30:29.448+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 7 ==== osd_ping(ping e59 up_from 9 ping_stamp 2026-02-11T23:30:29.450036+0000/20.303600311s send_stamp 20.303600311s delta_ub -1.980700016s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x396a000 2026-02-11T23:30:29.448+0000 7fac71bfd640 20 osd.0 60 get_con_osd_cluster to osd.1 from_epoch 60 2026-02-11T23:30:29.448+0000 7fac71bfd640 20 osd.0 60 get_nextmap_reserved map_reservations: {60=1} 2026-02-11T23:30:29.448+0000 7fac71bfd640 20 osd.0 60 release_map epoch: 60 2026-02-11T23:30:29.448+0000 7fac71bfd640 10 osd.0 60 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 59 -> 60 (shared) 2026-02-11T23:30:29.448+0000 7fac71bfd640 10 osd.0 60 send_incremental_map epoch range: (60, 60] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:30:29.448+0000 7fac71bfd640 10 osd.0 60 add_map_inc_bl 60 252 bytes 2026-02-11T23:30:29.448+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(60..60 src has 1..60) -- 0x46ea8c0 con 0x3752000 2026-02-11T23:30:29.448+0000 7fac723fe640 20 osd.0 60 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.981088042s,-1.980700016s]) 2026-02-11T23:30:29.448+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e60 up_from 5 ping_stamp 2026-02-11T23:30:29.450036+0000/20.303600311s send_stamp 22.284688950s delta_ub 1.981088042s) -- 0x48be000 con 0x396a000 2026-02-11T23:30:29.448+0000 7fac723fe640 20 osd.0 60 get_con_osd_cluster to osd.1 from_epoch 60 2026-02-11T23:30:29.448+0000 7fac723fe640 20 osd.0 60 get_nextmap_reserved map_reservations: {60=1} 2026-02-11T23:30:29.448+0000 7fac723fe640 20 osd.0 60 release_map epoch: 60 2026-02-11T23:30:29.448+0000 7fac723fe640 10 osd.0 60 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 60 is not newer than session's projected_epoch of 60 2026-02-11T23:30:29.578+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 38 ==== osd_map(61..62 src has 1..62) ==== 1200+0+0 (crc 0 0 0) 0x46eaa80 con 0x396bc00 2026-02-11T23:30:29.578+0000 7fac626e8640 20 osd.0 60 OSD::ms_dispatch: osd_map(61..62 src has 1..62) 2026-02-11T23:30:29.578+0000 7fac626e8640 20 osd.0 60 _dispatch 0x46eaa80 osd_map(61..62 src has 1..62) 2026-02-11T23:30:29.578+0000 7fac626e8640 3 osd.0 60 handle_osd_map epochs [61,62], i have 60, src has [1,62] 2026-02-11T23:30:29.578+0000 7fac626e8640 10 osd.0 60 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:29.578+0000 7fac626e8640 10 osd.0 60 handle_osd_map got inc map for epoch 61 2026-02-11T23:30:29.578+0000 7fac626e8640 20 osd.0 60 got_full_map 61, nothing requested 2026-02-11T23:30:29.578+0000 7fac626e8640 10 osd.0 60 handle_osd_map got inc map for epoch 62 2026-02-11T23:30:29.578+0000 7fac626e8640 20 osd.0 60 got_full_map 62, nothing requested 2026-02-11T23:30:29.578+0000 7fac626e8640 20 osd.0 60 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:29.578+0000 7fac626e8640 20 osd.0 60 handle_osd_map pg_num_history pg_num_history(e62 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1}} deleted_pools ) 2026-02-11T23:30:29.578+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {61={},62={}} 2026-02-11T23:30:29.578+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:29.578+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e62 maps [1~62] lci=[5,62] tlb=1) 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0 60 _committed_osd_maps 61..62 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0 60 advance to epoch 61 (<= last 62 <= newest_map 62) 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0 61 advance to epoch 62 (<= last 62 <= newest_map 62) 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 62 consume_map version 62 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 60 await_reserved_maps epoch:62 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 60 await_reserved_maps done 0 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 62 consume_map 62 -- publish done 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:6.0slot:0x25d91e0 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 62 identify_splits_and_merges 6.0 e60 to e62 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 62 identify_splits_and_merges 6.0 e60 to e62 pg_nums {59=1} 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 62 identify_splits_and_merges 4.0 e60 to e62 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 62 identify_splits_and_merges 4.0 e60 to e62 pg_nums {45=1} 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 62 identify_splits_and_merges 1.0 e60 to e62 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 62 identify_splits_and_merges 1.0 e60 to e62 pg_nums {24=1} 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 62 prune_pg_created 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0:0.consume_map 62 (was 60) 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 6.0 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0:1.consume_map 62 (was 60) 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0:2.consume_map 62 (was 60) 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0:3.consume_map 62 (was 60) 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0:4.consume_map 62 (was 60) 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0:5.consume_map 62 (was 60) 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0:6.consume_map 62 (was 60) 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0:7.consume_map 62 (was 60) 2026-02-11T23:30:29.578+0000 7fac5e983640 10 osd.0 62 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:29.578+0000 7fac5e983640 15 osd.0 62 enqueue_peering_evt 6.0 epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2 prio 255 cost 10 e62) 2026-02-11T23:30:29.578+0000 7fac5e983640 15 osd.0 62 enqueue_peering_evt 4.0 epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2 prio 255 cost 10 e62) 2026-02-11T23:30:29.578+0000 7fac5e983640 15 osd.0 62 enqueue_peering_evt 1.0 epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:29.578+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2 prio 255 cost 10 e62) 2026-02-11T23:30:29.578+0000 7fac5e983640 7 osd.0 62 activate_map version 62 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) queued 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.578+0000 7fac55971640 20 osd.0 62 agent_entry empty queue 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) pg 0x48c8000 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] handle_advance_map: 61 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) queued 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) pg 0x4856000 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0:0.update_pg_epoch 6.0 60 -> 61 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 60 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,2,0]/[4,2,0] -- 4/4 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 61 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 61 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 61 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 61 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] plpg_on_pool_change 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 60 -> 61 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 61 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] hit_set_clear 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 61 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] agent_stop 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 61 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] handle_advance_map: 62 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 61 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0:0.update_pg_epoch 6.0 61 -> 62 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 61 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 61 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,2]/[4,2] -- 4/4 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4,2] newacting [4,2] 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:29.578+0000 7fac55170640 5 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.909120 6 0.000102 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 61 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 62 2026-02-11T23:30:29.578+0000 7fac55170640 5 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] exit Started/ReplicaActive 0.909125 0 0.000000 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 61 -> 62 2026-02-11T23:30:29.578+0000 7fac55170640 5 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] exit Started 0.909268 0 0.000000 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 61 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:29.578+0000 7fac55170640 5 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=59 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 62 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=62 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=62 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 61..62 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=62 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=62 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4,2] newacting [4,2] 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=62 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 62 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2,0] r=2 lpr=62 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 62 2026-02-11T23:30:29.578+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:29.578+0000 7fac55170640 15 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2] r=2 lpr=62 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:29.578+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(59-61 up [4, 2, 0](4) acting [4, 2, 0](4)) up_thru 59 up_from 21 last_epoch_clean 60 interval(59-61 up [4, 2, 0](4) acting [4, 2, 0](4) maybe_went_rw) : primary up 21-59 includes interval 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) queued 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 NullEvt) class_id 2immediate prio 255 cost 10 e62) pg 0x40f8000 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=59) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([59,61] all_participants={0, 2, 4} intervals=[([59,61] acting={0, 2, 4})]) 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 61 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 60 -> 61 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4,2]+[4,2] 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 60 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 61 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 61 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active pruub 37.559883118s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 37.559883118s (mnow 22.414527893s + 15.145355225s) 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 61 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 62 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 61 -> 62 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active pruub 37.559883118s@ mbc={}] cancel_recovery 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active pruub 37.559883118s@ mbc={}] clear_recovery_state 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 61 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:29.578+0000 7fac55170640 15 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active pruub 37.559883118s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active pruub 37.559883118s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:29.578+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active pruub 37.559883118s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active pruub 37.559883118s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 61..62 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active pruub 37.559883118s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:29.578+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 62 2026-02-11T23:30:29.578+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:29.578+0000 7fac55170640 1 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 active pruub 37.559883118s@ mbc={}] PeeringState::start_peering_interval up [4,2,0] -> [4,2], acting [4,2,0] -> [4,2], acting_primary 4 -> 4, up_primary 4 -> 4, role 2 -> -1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:29.578+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:29.578+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] clear_primary_state 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] release_backoffs [6:00000000::::0,MAX) 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:29.578+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:29.578+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] agent_stop 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] on_change 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] on_change_cleanup 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] on_change 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] exit NotTrimming 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown pruub 37.559883118s@ mbc={}] enter NotTrimming 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] plpg_on_role_change 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] hit_set_clear 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] agent_stop 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 62 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] handle_activate_map: epoch range: 61..62 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145355225s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:29.578+0000 7fac55170640 5 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145273209s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] exit Reset 0.000133 1 0.000130 2026-02-11T23:30:29.578+0000 7fac55170640 5 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145273209s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] enter Started 2026-02-11T23:30:29.578+0000 7fac55170640 5 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145273209s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] enter Start 2026-02-11T23:30:29.578+0000 7fac55170640 1 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145273209s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:29.578+0000 7fac55170640 5 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145273209s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] exit Start 0.000003 0 0.000000 2026-02-11T23:30:29.578+0000 7fac55170640 5 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145273209s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] enter Started/Stray 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145273209s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 60 while current is 62 2026-02-11T23:30:29.578+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145273209s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145273209s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] do_peering_event: epoch_sent: 62 epoch_requested: 62 NullEvt 2026-02-11T23:30:29.578+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 62 get_con_osd_cluster to osd.4 from_epoch 62 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 62 get_nextmap_reserved map_reservations: {62=1} 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 62 release_map epoch: 62 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 62 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 60 -> 62 (shared) 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 62 send_incremental_map epoch range: (61, 62] to v2:127.0.0.1:6836/864727417 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 62 add_map_inc_bl 61 574 bytes 2026-02-11T23:30:29.578+0000 7fac55170640 10 osd.0 62 add_map_inc_bl 62 574 bytes 2026-02-11T23:30:29.578+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(61..62 src has 1..62) -- 0x3940c40 con 0x3e4a000 2026-02-11T23:30:29.578+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(6.0 6.0 (query:62 sent:62 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145273209s) ([59,61] all_participants={0, 2, 4} intervals=[([59,61] acting={0, 2, 4})])) e62/62) -- 0x4869500 con 0x3e4a000 2026-02-11T23:30:29.578+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.578+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.579+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.579+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.579+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 25 ==== pg_query2(6.0 6.0 query(info 0'0 epoch_sent 62) e62/62) ==== 158+0+0 (unknown 0 0 0) 0x3f3f680 con 0x3e4a000 2026-02-11T23:30:29.579+0000 7fac703fa640 15 osd.0 62 enqueue_peering_evt 6.0 epoch_sent: 62 epoch_requested: 62 MQuery 6.0 from 4 query_epoch 62 query: query(info 0'0 epoch_sent 62) 2026-02-11T23:30:29.579+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 24 ==== osd_map(61..62 src has 1..62) ==== 1200+0+0 (unknown 0 0 0) 0x3940c40 con 0x3e4a000 2026-02-11T23:30:29.579+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 MQuery 6.0 from 4 query_epoch 62 query: query(info 0'0 epoch_sent 62)) class_id 2 prio 255 cost 10 e62) 2026-02-11T23:30:29.579+0000 7fac606e4640 20 osd.0 62 OSD::ms_dispatch: osd_map(61..62 src has 1..62) 2026-02-11T23:30:29.579+0000 7fac606e4640 20 osd.0 62 _dispatch 0x3940c40 osd_map(61..62 src has 1..62) 2026-02-11T23:30:29.579+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.579+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 MQuery 6.0 from 4 query_epoch 62 query: query(info 0'0 epoch_sent 62)) class_id 2immediate prio 255 cost 10 e62) queued 2026-02-11T23:30:29.579+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.579+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 62 epoch_requested: 62 MQuery 6.0 from 4 query_epoch 62 query: query(info 0'0 epoch_sent 62)) class_id 2immediate prio 255 cost 10 e62) pg 0x48c8000 2026-02-11T23:30:29.579+0000 7fac606e4640 3 osd.0 62 handle_osd_map epochs [61,62], i have 62, src has [1,62] 2026-02-11T23:30:29.579+0000 7fac606e4640 10 osd.0 62 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:29.579+0000 7fac606e4640 10 osd.0 62 no new maps here, dropping 2026-02-11T23:30:29.579+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.145273209s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] do_peering_event: epoch_sent: 62 epoch_requested: 62 MQuery 6.0 from 4 query_epoch 62 query: query(info 0'0 epoch_sent 62) 2026-02-11T23:30:29.579+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559883118s@ mbc={}] PeeringState::update_history advanced history from ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s 2026-02-11T23:30:29.579+0000 7fac55170640 20 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] PeeringState::update_history prior_readable_until_ub 37.559852600s (mnow 22.415771484s + 15.144081116s) 2026-02-11T23:30:29.579+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:29.579+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.579+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:29.579+0000 7fac55170640 20 osd.0 62 get_con_osd_cluster to osd.4 from_epoch 62 2026-02-11T23:30:29.579+0000 7fac55170640 20 osd.0 62 get_nextmap_reserved map_reservations: {62=1} 2026-02-11T23:30:29.579+0000 7fac55170640 20 osd.0 62 release_map epoch: 62 2026-02-11T23:30:29.579+0000 7fac55170640 10 osd.0 62 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 62 is not newer than session's projected_epoch of 62 2026-02-11T23:30:29.579+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(6.0 6.0 (query:62 sent:62 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) ([59,61] all_participants={0, 2, 4} intervals=[([59,61] acting={0, 2, 4})])) e62/62) -- 0x486a300 con 0x3e4a000 2026-02-11T23:30:29.579+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.580+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.580+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.706+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 26 ==== osd pg remove(epoch 63; pg6.0; ) ==== 32+0+0 (unknown 0 0 0) 0x2580160 con 0x3e4a000 2026-02-11T23:30:29.706+0000 7fac703fa640 7 osd.0 62 handle_fast_pg_remove osd pg remove(epoch 63; pg6.0; ) from osd.4 2026-02-11T23:30:29.706+0000 7fac703fa640 15 osd.0 62 enqueue_peering_evt 6.0 epoch_sent: 63 epoch_requested: 63 DeleteStart 2026-02-11T23:30:29.706+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 DeleteStart) class_id 2 prio 255 cost 10 e63) 2026-02-11T23:30:29.706+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.706+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 DeleteStart) class_id 2immediate prio 255 cost 10 e63) queued 2026-02-11T23:30:29.706+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.706+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 DeleteStart) class_id 2immediate prio 255 cost 10 e63) pg 0x48c8000 2026-02-11T23:30:29.706+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 6.0 peering, item epoch is 63, will wait on OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 DeleteStart) class_id 2immediate prio 255 cost 10 e63) 2026-02-11T23:30:29.706+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.706+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.984+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 39 ==== osd_map(64..65 src has 1..65) ==== 1200+0+0 (crc 0 0 0) 0x3941500 con 0x396bc00 2026-02-11T23:30:29.984+0000 7fac626e8640 20 osd.0 62 OSD::ms_dispatch: osd_map(64..65 src has 1..65) 2026-02-11T23:30:29.984+0000 7fac626e8640 20 osd.0 62 _dispatch 0x3941500 osd_map(64..65 src has 1..65) 2026-02-11T23:30:29.984+0000 7fac626e8640 10 monclient: _renew_subs 2026-02-11T23:30:29.984+0000 7fac626e8640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:29.984+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=63}) -- 0x449d860 con 0x396bc00 2026-02-11T23:30:29.984+0000 7fac626e8640 3 osd.0 62 handle_osd_map epochs [64,65], i have 62, src has [1,65] 2026-02-11T23:30:29.984+0000 7fac626e8640 10 osd.0 62 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:29.984+0000 7fac626e8640 10 osd.0 62 handle_osd_map message skips epochs 63..63 2026-02-11T23:30:29.985+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 40 ==== osd_map(63..65 src has 1..65) ==== 1790+0+0 (crc 0 0 0) 0x39416c0 con 0x396bc00 2026-02-11T23:30:29.985+0000 7fac626e8640 20 osd.0 62 OSD::ms_dispatch: osd_map(63..65 src has 1..65) 2026-02-11T23:30:29.985+0000 7fac626e8640 20 osd.0 62 _dispatch 0x39416c0 osd_map(63..65 src has 1..65) 2026-02-11T23:30:29.985+0000 7fac626e8640 3 osd.0 62 handle_osd_map epochs [63,65], i have 62, src has [1,65] 2026-02-11T23:30:29.985+0000 7fac626e8640 10 osd.0 62 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:29.985+0000 7fac626e8640 10 osd.0 62 handle_osd_map got inc map for epoch 63 2026-02-11T23:30:29.985+0000 7fac626e8640 20 osd.0 62 got_full_map 63, nothing requested 2026-02-11T23:30:29.985+0000 7fac626e8640 10 osd.0 62 handle_osd_map got inc map for epoch 64 2026-02-11T23:30:29.985+0000 7fac626e8640 20 osd.0 62 got_full_map 64, nothing requested 2026-02-11T23:30:29.985+0000 7fac626e8640 10 osd.0 62 handle_osd_map got inc map for epoch 65 2026-02-11T23:30:29.985+0000 7fac626e8640 20 osd.0 62 got_full_map 65, nothing requested 2026-02-11T23:30:29.985+0000 7fac626e8640 20 osd.0 62 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:29.985+0000 7fac626e8640 20 osd.0 62 handle_osd_map pg_num_history pg_num_history(e65 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1}} deleted_pools ) 2026-02-11T23:30:29.985+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {63={},64={},65={}} 2026-02-11T23:30:29.985+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:29.985+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e65 maps [1~65] lci=[5,65] tlb=1) 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0 62 _committed_osd_maps 63..65 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0 62 advance to epoch 63 (<= last 65 <= newest_map 65) 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0 63 advance to epoch 64 (<= last 65 <= newest_map 65) 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0 64 advance to epoch 65 (<= last 65 <= newest_map 65) 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 65 consume_map version 65 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 62 await_reserved_maps epoch:65 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 62 await_reserved_maps done 0 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 65 consume_map 65 -- publish done 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:6.0slot:0x25d91e0 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 65 identify_splits_and_merges 6.0 e62 to e65 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 65 identify_splits_and_merges 6.0 e62 to e65 pg_nums {59=1} 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 65 identify_splits_and_merges 4.0 e62 to e65 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 65 identify_splits_and_merges 4.0 e62 to e65 pg_nums {45=1} 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 65 identify_splits_and_merges 1.0 e62 to e65 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 65 identify_splits_and_merges 1.0 e62 to e65 pg_nums {24=1} 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 65 prune_pg_created 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0:0.consume_map 65 (was 62) 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 6.0 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 6.0 pending_peering first epoch 63 <= 65, requeueing 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 6.0 to_process <> waiting <> waiting_peering {63=} 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0:1.consume_map 65 (was 62) 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0:2.consume_map 65 (was 62) 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0:3.consume_map 65 (was 62) 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0:4.consume_map 65 (was 62) 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0:5.consume_map 65 (was 62) 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0:6.consume_map 65 (was 62) 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0:7.consume_map 65 (was 62) 2026-02-11T23:30:29.985+0000 7fac5e983640 10 osd.0 65 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.985+0000 7fac5e983640 15 osd.0 65 enqueue_peering_evt 6.0 epoch_sent: 65 epoch_requested: 65 NullEvt 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 DeleteStart) class_id 2immediate prio 255 cost 10 e63) queued 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 63 epoch_requested: 63 DeleteStart) class_id 2immediate prio 255 cost 10 e63) pg 0x48c8000 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2 prio 255 cost 10 e65) 2026-02-11T23:30:29.985+0000 7fac5e983640 15 osd.0 65 enqueue_peering_evt 4.0 epoch_sent: 65 epoch_requested: 65 NullEvt 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2 prio 255 cost 10 e65) 2026-02-11T23:30:29.985+0000 7fac5e983640 15 osd.0 65 enqueue_peering_evt 1.0 epoch_sent: 65 epoch_requested: 65 NullEvt 2026-02-11T23:30:29.985+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2 prio 255 cost 10 e65) 2026-02-11T23:30:29.985+0000 7fac5e983640 7 osd.0 65 activate_map version 65 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] handle_advance_map: 63 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0:0.update_pg_epoch 6.0 62 -> 63 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] PeeringState::advance_map handle_advance_map [4,2]/[4,2] -- 4/4 2026-02-11T23:30:29.985+0000 7fac55971640 20 osd.0 65 agent_entry empty queue 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 63 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] state: Started advmap 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 63 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 63 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 63 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] hit_set_clear 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2immediate prio 255 cost 10 e65) queued 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 63 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] agent_stop 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 63 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] handle_advance_map: 64 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0:0.update_pg_epoch 6.0 63 -> 64 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 63 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] PeeringState::advance_map handle_advance_map [4,2]/[4,2] -- 4/4 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] state: Started advmap 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] hit_set_clear 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] agent_stop 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] handle_advance_map: 65 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0:0.update_pg_epoch 6.0 64 -> 65 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 64 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] PeeringState::advance_map handle_advance_map [4,2]/[4,2] -- 4/4 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] state: Started advmap 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] hit_set_clear 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] agent_stop 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] handle_activate_map: epoch range: 63..65 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=15.144081116s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 62 while current is 65 2026-02-11T23:30:29.985+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] do_peering_event: epoch_sent: 63 epoch_requested: 63 DeleteStart 2026-02-11T23:30:29.985+0000 7fac55170640 5 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] exit Started/Stray 0.406833 7 0.000049 2026-02-11T23:30:29.985+0000 7fac55170640 5 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] enter Started/ToDelete 2026-02-11T23:30:29.985+0000 7fac55170640 5 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] enter Started/ToDelete/WaitDeleteReserved 2026-02-11T23:30:29.985+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 65 get_con_osd_cluster to osd.4 from_epoch 65 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 65 get_nextmap_reserved map_reservations: {65=1} 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 65 release_map epoch: 65 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 65 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 62 -> 65 (shared) 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 65 send_incremental_map epoch range: (63, 65] to v2:127.0.0.1:6836/864727417 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 65 add_map_inc_bl 63 582 bytes 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 65 add_map_inc_bl 64 574 bytes 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 65 add_map_inc_bl 65 574 bytes 2026-02-11T23:30:29.985+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(63..65 src has 1..65) -- 0x46eafc0 con 0x3e4a000 2026-02-11T23:30:29.985+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(6.0 6.0 (query:65 sent:65 6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) ([59,61] all_participants={0, 2, 4} intervals=[([59,61] acting={0, 2, 4})])) e65/65) -- 0x486aa00 con 0x3e4a000 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2immediate prio 255 cost 10 e65) queued 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2immediate prio 255 cost 10 e65) pg 0x4856000 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2immediate prio 255 cost 10 e65) pg 0x48c8000 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 63 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 62 -> 63 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 62 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] do_peering_event: epoch_sent: 65 epoch_requested: 65 NullEvt 2026-02-11T23:30:29.985+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 63 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 63 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2immediate prio 255 cost 10 e65) queued 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 63 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 64 2026-02-11T23:30:29.985+0000 7fac58176640 15 osd.0 65 enqueue_peering_evt 6.0 epoch_sent: 65 epoch_requested: 65 DeleteReserved 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.985+0000 7fac58176640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 DeleteReserved) class_id 2 prio 255 cost 10 e65) 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 NullEvt) class_id 2immediate prio 255 cost 10 e65) pg 0x40f8000 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 63 -> 64 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 63 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 63 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 62 -> 63 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 62 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 65 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 64 -> 65 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 63 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 63 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 64 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 63 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 64 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 63 -> 64 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 63 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 63..65 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 65 2026-02-11T23:30:29.985+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 65 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 64 -> 65 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 64 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 65 epoch_requested: 65 NullEvt 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:29.985+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 65 advance_pg new pool opts old pool opts 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 63..65 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 DeleteReserved) class_id 2immediate prio 255 cost 10 e65) queued 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 65 2026-02-11T23:30:29.985+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGPeeringEvent(epoch_sent: 65 epoch_requested: 65 DeleteReserved) class_id 2immediate prio 255 cost 10 e65) pg 0x48c8000 2026-02-11T23:30:29.985+0000 7fac51168640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 65 epoch_requested: 65 NullEvt 2026-02-11T23:30:29.985+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:29.985+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] do_peering_event: epoch_sent: 65 epoch_requested: 65 DeleteReserved 2026-02-11T23:30:29.985+0000 7fac55170640 5 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] exit Started/ToDelete/WaitDeleteReserved 0.000250 2 0.000016 2026-02-11T23:30:29.985+0000 7fac55170640 5 osd.0 pg_epoch: 65 pg[6.0( empty local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] enter Started/ToDelete/Deleting 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] on_removal 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] on_shutdown 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:29.985+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:29.985+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] clear_backoffs 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] exit NotTrimming 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] enter NotTrimming 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] on_change 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] clear_async_reads 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] clear_primary_state 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] release_backoffs [6:00000000::::0,MAX) 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:29.985+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:29.985+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] agent_stop 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] cancel_recovery 2026-02-11T23:30:29.985+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] clear_recovery_state 2026-02-11T23:30:29.985+0000 7fac55170640 15 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:29.985+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:29.985+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.986+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.986+0000 7fac55170640 10 osd.0 65 queue_for_pg_delete on 6.0 e 65 2026-02-11T23:30:29.986+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(6.0 PGDelete(pgid=6.0 epoch_queued=65) class_id 1 prio 5 cost 1048576 e65) 2026-02-11T23:30:29.986+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:29.986+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGDelete(6.0 e65) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e65) queued 2026-02-11T23:30:29.986+0000 7fac55170640 20 osd.0 op_wq(0) _process 6.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:29.986+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(6.0 PGDelete(6.0 e65) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e65) pg 0x48c8000 2026-02-11T23:30:29.986+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.986+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] do_peering_event: epoch_sent: 65 epoch_requested: 65 DeleteSome 2026-02-11T23:30:29.986+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] do_delete_work 2026-02-11T23:30:29.986+0000 7fac55170640 20 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] do_delete_work [#6:00000000::::head#] 2026-02-11T23:30:29.986+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.986+0000 7fac55170640 20 osd.0 65 try_finish_pg_delete 6.0 0x48c8000 2026-02-11T23:30:29.986+0000 7fac55170640 10 osd.0:0._detach_pg 6.0 0x48c8000 2026-02-11T23:30:29.986+0000 7fac55170640 5 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 DELETING pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] exit Started/ToDelete/Deleting 0.000375 1 0.000044 2026-02-11T23:30:29.986+0000 7fac55170640 5 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] exit Started/ToDelete 0.000634 0 0.000000 2026-02-11T23:30:29.986+0000 7fac55170640 5 osd.0 pg_epoch: 65 pg[6.0( empty lb MIN local-lis/les=59/60 n=0 ec=59/59 lis/c=59/59 les/c/f=60/60/0 sis=62 pruub=14.738408089s) [4,2] r=-1 lpr=62 pi=[59,62)/1 crt=0'0 unknown NOTIFY pruub 37.559852600s@ mbc={}] exit Started 0.407483 0 0.000000 2026-02-11T23:30:29.986+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:29.986+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.986+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:29.986+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:29.986+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:29.986+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.016+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 4 ==== osd_ping(ping e63 up_from 13 ping_stamp 2026-02-11T23:30:30.017833+0000/18.203227997s send_stamp 18.203227997s delta_ub -4.648861408s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a300 con 0x3f56400 2026-02-11T23:30:30.016+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 4 ==== osd_ping(ping e63 up_from 13 ping_stamp 2026-02-11T23:30:30.017833+0000/18.203227997s send_stamp 18.203227997s delta_ub -4.648861408s) ==== 2033+0+0 (crc 0 0 0) 0x48be000 con 0x3f56000 2026-02-11T23:30:30.016+0000 7fac703fa640 20 osd.0 65 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649099827s,-4.648861408s]) 2026-02-11T23:30:30.016+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e65 up_from 5 ping_stamp 2026-02-11T23:30:30.017833+0000/18.203227997s send_stamp 22.852329254s delta_ub 4.649099827s) -- 0x3b1b680 con 0x3f56400 2026-02-11T23:30:30.016+0000 7fac703fa640 20 osd.0 65 get_con_osd_cluster to osd.2 from_epoch 65 2026-02-11T23:30:30.016+0000 7fac703fa640 20 osd.0 65 get_nextmap_reserved map_reservations: {65=1} 2026-02-11T23:30:30.016+0000 7fac703fa640 20 osd.0 65 release_map epoch: 65 2026-02-11T23:30:30.016+0000 7fac703fa640 10 osd.0 65 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 59 to ping map epoch of 63 2026-02-11T23:30:30.016+0000 7fac703fa640 10 osd.0 65 maybe_share_map: con v2:127.0.0.1:6820/2662569638 map epoch 63 -> 65 (shared) 2026-02-11T23:30:30.016+0000 7fac703fa640 10 osd.0 65 send_incremental_map epoch range: (64, 65] to v2:127.0.0.1:6820/2662569638 2026-02-11T23:30:30.016+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] -- osd_map(64..65 src has 1..65) -- 0x46eafc0 con 0x3f57c00 2026-02-11T23:30:30.016+0000 7fac723fe640 20 osd.0 65 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649135590s,-4.648861408s]) 2026-02-11T23:30:30.016+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e65 up_from 5 ping_stamp 2026-02-11T23:30:30.017833+0000/18.203227997s send_stamp 22.852363586s delta_ub 4.649135590s) -- 0x3b1b380 con 0x3f56000 2026-02-11T23:30:30.016+0000 7fac723fe640 20 osd.0 65 get_con_osd_cluster to osd.2 from_epoch 65 2026-02-11T23:30:30.016+0000 7fac723fe640 20 osd.0 65 get_nextmap_reserved map_reservations: {65=1} 2026-02-11T23:30:30.016+0000 7fac723fe640 20 osd.0 65 release_map epoch: 65 2026-02-11T23:30:30.016+0000 7fac723fe640 10 osd.0 65 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 65 is not newer than session's projected_epoch of 65 2026-02-11T23:30:30.089+0000 7fac68beb640 10 osd.0 65 tick 2026-02-11T23:30:30.089+0000 7fac68beb640 20 osd.0 65 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:30.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:30.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:30.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:00.168592+0000) 2026-02-11T23:30:30.167+0000 7fac616e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:30.188+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 41 ==== osd_map(65..66 src has 1..66) ==== 1217+0+0 (crc 0 0 0) 0x3941180 con 0x396bc00 2026-02-11T23:30:30.188+0000 7fac626e8640 20 osd.0 65 OSD::ms_dispatch: osd_map(65..66 src has 1..66) 2026-02-11T23:30:30.188+0000 7fac626e8640 20 osd.0 65 _dispatch 0x3941180 osd_map(65..66 src has 1..66) 2026-02-11T23:30:30.188+0000 7fac626e8640 3 osd.0 65 handle_osd_map epochs [65,66], i have 65, src has [1,66] 2026-02-11T23:30:30.188+0000 7fac626e8640 10 osd.0 65 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:30.188+0000 7fac626e8640 10 osd.0 65 handle_osd_map got inc map for epoch 66 2026-02-11T23:30:30.188+0000 7fac626e8640 20 osd.0 65 got_full_map 66, nothing requested 2026-02-11T23:30:30.188+0000 7fac626e8640 10 osd.0 65 _track_pools_and_pg_num_changes recording new pool 7 pg_num 1 2026-02-11T23:30:30.188+0000 7fac626e8640 20 osd.0 65 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:30.188+0000 7fac626e8640 20 osd.0 65 handle_osd_map pg_num_history pg_num_history(e66 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1}} deleted_pools ) 2026-02-11T23:30:30.188+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {66={}} 2026-02-11T23:30:30.188+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:30.188+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e66 maps [1~66] lci=[5,66] tlb=1) 2026-02-11T23:30:30.188+0000 7fac5e983640 10 osd.0 65 _committed_osd_maps 66..66 2026-02-11T23:30:30.188+0000 7fac5e983640 10 osd.0 65 advance to epoch 66 (<= last 66 <= newest_map 66) 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0 66 consume_map version 66 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0 65 await_reserved_maps epoch:66 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0 65 await_reserved_maps done 0 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0 66 consume_map 66 -- publish done 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:6.0slot:0x25d91e0 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 6.0 has no pg and waiting_for_split 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0 66 identify_splits_and_merges 4.0 e65 to e66 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0 66 identify_splits_and_merges 4.0 e65 to e66 pg_nums {45=1} 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0 66 identify_splits_and_merges 1.0 e65 to e66 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0 66 identify_splits_and_merges 1.0 e65 to e66 pg_nums {24=1} 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0 66 prune_pg_created 2026-02-11T23:30:30.188+0000 7fac5e983640 10 osd.0:0.consume_map 66 (was 65) 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 6.0 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 6.0 empty, pruning 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:30.188+0000 7fac5e983640 10 osd.0:1.consume_map 66 (was 65) 2026-02-11T23:30:30.188+0000 7fac5e983640 10 osd.0:2.consume_map 66 (was 65) 2026-02-11T23:30:30.188+0000 7fac5e983640 10 osd.0:3.consume_map 66 (was 65) 2026-02-11T23:30:30.188+0000 7fac5e983640 10 osd.0:4.consume_map 66 (was 65) 2026-02-11T23:30:30.188+0000 7fac5e983640 10 osd.0:5.consume_map 66 (was 65) 2026-02-11T23:30:30.188+0000 7fac5e983640 10 osd.0:6.consume_map 66 (was 65) 2026-02-11T23:30:30.188+0000 7fac5e983640 10 osd.0:7.consume_map 66 (was 65) 2026-02-11T23:30:30.188+0000 7fac5e983640 10 osd.0 66 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:30.188+0000 7fac5e983640 15 osd.0 66 enqueue_peering_evt 4.0 epoch_sent: 66 epoch_requested: 66 NullEvt 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2 prio 255 cost 10 e66) 2026-02-11T23:30:30.188+0000 7fac5e983640 15 osd.0 66 enqueue_peering_evt 1.0 epoch_sent: 66 epoch_requested: 66 NullEvt 2026-02-11T23:30:30.188+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2 prio 255 cost 10 e66) 2026-02-11T23:30:30.188+0000 7fac5e983640 7 osd.0 66 activate_map version 66 2026-02-11T23:30:30.188+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.188+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2immediate prio 255 cost 10 e66) queued 2026-02-11T23:30:30.188+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.188+0000 7fac55971640 20 osd.0 66 agent_entry empty queue 2026-02-11T23:30:30.188+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2immediate prio 255 cost 10 e66) pg 0x4856000 2026-02-11T23:30:30.188+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:30.188+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2immediate prio 255 cost 10 e66) queued 2026-02-11T23:30:30.188+0000 7fac51168640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 66 2026-02-11T23:30:30.188+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 65 -> 66 2026-02-11T23:30:30.188+0000 7fac51168640 10 osd.0 pg_epoch: 65 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:30.188+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:30.188+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 66 epoch_requested: 66 NullEvt) class_id 2immediate prio 255 cost 10 e66) pg 0x40f8000 2026-02-11T23:30:30.188+0000 7fac51168640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:30.188+0000 7fac51168640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:30.188+0000 7fac51168640 20 osd.0 66 advance_pg new pool opts old pool opts 2026-02-11T23:30:30.188+0000 7fac51168640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 66..66 2026-02-11T23:30:30.188+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 66 2026-02-11T23:30:30.188+0000 7fac51168640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:30.188+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 65 -> 66 2026-02-11T23:30:30.188+0000 7fac51168640 20 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 66 2026-02-11T23:30:30.188+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:30.188+0000 7fac55170640 10 osd.0 pg_epoch: 65 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:30.188+0000 7fac51168640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 66 epoch_requested: 66 NullEvt 2026-02-11T23:30:30.188+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:30.188+0000 7fac55170640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:30.188+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.188+0000 7fac55170640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:30.188+0000 7fac55170640 20 osd.0 66 advance_pg new pool opts old pool opts 2026-02-11T23:30:30.188+0000 7fac55170640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 66..66 2026-02-11T23:30:30.188+0000 7fac55170640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:30.188+0000 7fac55170640 20 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 66 2026-02-11T23:30:30.188+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:30.188+0000 7fac55170640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 66 epoch_requested: 66 NullEvt 2026-02-11T23:30:30.188+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:30.188+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:30.254+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 6 ==== osd_ping(ping e65 up_from 21 ping_stamp 2026-02-11T23:30:30.256183+0000/13.203689575s send_stamp 13.203689575s delta_ub -9.886499405s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b680 con 0x3f57400 2026-02-11T23:30:30.254+0000 7fac703fa640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886982918s,-9.886499405s]) 2026-02-11T23:30:30.254+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e66 up_from 5 ping_stamp 2026-02-11T23:30:30.256183+0000/13.203689575s send_stamp 23.090673447s delta_ub 9.886982918s) -- 0x3b1a300 con 0x3f57400 2026-02-11T23:30:30.254+0000 7fac703fa640 20 osd.0 66 get_con_osd_cluster to osd.4 from_epoch 66 2026-02-11T23:30:30.254+0000 7fac703fa640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:30.254+0000 7fac703fa640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:30.254+0000 7fac703fa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 65 -> 66 (shared) 2026-02-11T23:30:30.254+0000 7fac703fa640 10 osd.0 66 send_incremental_map epoch range: (66, 66] to v2:127.0.0.1:6836/864727417 2026-02-11T23:30:30.254+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 6 ==== osd_ping(ping e65 up_from 21 ping_stamp 2026-02-11T23:30:30.256183+0000/13.203689575s send_stamp 13.203689575s delta_ub -9.886499405s) ==== 2033+0+0 (crc 0 0 0) 0x3b1af00 con 0x3f57800 2026-02-11T23:30:30.254+0000 7fac703fa640 10 osd.0 66 add_map_inc_bl 66 591 bytes 2026-02-11T23:30:30.254+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(66..66 src has 1..66) -- 0x46ea8c0 con 0x3e4a000 2026-02-11T23:30:30.254+0000 7fac71bfd640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887076378s,-9.886499405s]) 2026-02-11T23:30:30.254+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e66 up_from 5 ping_stamp 2026-02-11T23:30:30.256183+0000/13.203689575s send_stamp 23.090764999s delta_ub 9.887076378s) -- 0x3b1ac00 con 0x3f57800 2026-02-11T23:30:30.254+0000 7fac71bfd640 20 osd.0 66 get_con_osd_cluster to osd.4 from_epoch 66 2026-02-11T23:30:30.254+0000 7fac71bfd640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:30.254+0000 7fac71bfd640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:30.254+0000 7fac71bfd640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 66 is not newer than session's projected_epoch of 66 2026-02-11T23:30:30.294+0000 7fac683ea640 10 osd.0 66 tick_without_osd_lock 2026-02-11T23:30:30.294+0000 7fac683ea640 0 osd.0 66 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:30.294+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:30.294+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:30.294+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:30.294+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.140 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:30.294+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:30.296, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:30.294+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:30.294+0000 7fac683ea640 10 osd.0 66 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:30.294+0000 7fac683ea640 20 osd.0 66 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:30.294+0000 7fac683ea640 10 osd.0 66 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:30.294+0000 7fac683ea640 20 osd.0 66 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:31.048+0000 7fac68beb640 10 osd.0 66 tick 2026-02-11T23:30:31.048+0000 7fac68beb640 20 osd.0 66 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:31.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:31.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:31.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:01.168670+0000) 2026-02-11T23:30:31.320+0000 7fac683ea640 10 osd.0 66 tick_without_osd_lock 2026-02-11T23:30:31.320+0000 7fac683ea640 0 osd.0 66 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:31.320+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:31.320+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:31.320+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:31.320+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.140 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:31.320+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:31.321, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:31.320+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:31.320+0000 7fac683ea640 10 osd.0 66 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:31.320+0000 7fac683ea640 20 osd.0 66 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:31.320+0000 7fac683ea640 10 osd.0 66 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:31.320+0000 7fac683ea640 20 osd.0 66 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:31.478+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 4 ==== osd_ping(ping e66 up_from 17 ping_stamp 2026-02-11T23:30:31.479588+0000/17.003623962s send_stamp 17.003623962s delta_ub -7.310010433s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ac00 con 0x3f56800 2026-02-11T23:30:31.478+0000 7fac71bfd640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310488224s,-7.310010433s]) 2026-02-11T23:30:31.478+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e66 up_from 5 ping_stamp 2026-02-11T23:30:31.479588+0000/17.003623962s send_stamp 24.314111710s delta_ub 7.310488224s) -- 0x3b1af00 con 0x3f56800 2026-02-11T23:30:31.478+0000 7fac71bfd640 20 osd.0 66 get_con_osd_cluster to osd.3 from_epoch 66 2026-02-11T23:30:31.478+0000 7fac71bfd640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:31.478+0000 7fac71bfd640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:31.478+0000 7fac71bfd640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 60 to ping map epoch of 66 2026-02-11T23:30:31.478+0000 7fac71bfd640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 66 is not newer than session's projected_epoch of 66 2026-02-11T23:30:31.478+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 4 ==== osd_ping(ping e66 up_from 17 ping_stamp 2026-02-11T23:30:31.479588+0000/17.003623962s send_stamp 17.003623962s delta_ub -7.310010433s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x3f56c00 2026-02-11T23:30:31.478+0000 7fac723fe640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310511589s,-7.310010433s]) 2026-02-11T23:30:31.478+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e66 up_from 5 ping_stamp 2026-02-11T23:30:31.479588+0000/17.003623962s send_stamp 24.314136505s delta_ub 7.310511589s) -- 0x48be000 con 0x3f56c00 2026-02-11T23:30:31.478+0000 7fac723fe640 20 osd.0 66 get_con_osd_cluster to osd.3 from_epoch 66 2026-02-11T23:30:31.478+0000 7fac723fe640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:31.478+0000 7fac723fe640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:31.478+0000 7fac723fe640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 66 is not newer than session's projected_epoch of 66 2026-02-11T23:30:31.568+0000 7fac4d160640 5 osd.0 66 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x4082d/0x9c000, compress 0x0/0x0/0x0, omap 0x7127, meta 0x1a48ed9), peers [1,2,3,4] op hist []) 2026-02-11T23:30:31.568+0000 7fac4d160640 20 osd.0 66 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:31.568+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 24.404333115s delta_ub 1.981088042s) -- 0x3b1b080 con 0x396b400 2026-02-11T23:30:31.568+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 24.404333115s delta_ub 1.981088042s) -- 0x3b1ad80 con 0x3752400 2026-02-11T23:30:31.568+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 24.404333115s delta_ub 4.649135590s) -- 0x3b1b200 con 0x396b800 2026-02-11T23:30:31.568+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 24.404333115s delta_ub 4.649135590s) -- 0x3b1a000 con 0x34c1000 2026-02-11T23:30:31.568+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 24.404333115s delta_ub 7.310511589s) -- 0x3b1b800 con 0x3740000 2026-02-11T23:30:31.568+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 24.404333115s delta_ub 7.310511589s) -- 0x3b1bb00 con 0x3740400 2026-02-11T23:30:31.568+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 24.404333115s delta_ub 9.887076378s) -- 0x3b1be00 con 0x3740800 2026-02-11T23:30:31.568+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e66 up_from 5 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 24.404333115s delta_ub 9.887076378s) -- 0x3b1a480 con 0x3740c00 2026-02-11T23:30:31.568+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 7 ==== osd_ping(ping_reply e68 up_from 9 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 22.423538208s delta_ub -1.980795622s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bc80 con 0x396b400 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 get_con_osd_cluster to osd.1 from_epoch 66 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:31.568+0000 7fac703fa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 60 to ping map epoch of 68 2026-02-11T23:30:31.568+0000 7fac703fa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 66 is not newer than session's projected_epoch of 68 2026-02-11T23:30:31.568+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 6 ==== osd_ping(ping_reply e68 up_from 13 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 19.755397797s delta_ub -4.648936749s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x396b800 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980929017s,-1.980795622s]) 2026-02-11T23:30:31.568+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 7 ==== osd_ping(ping_reply e68 up_from 9 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 22.423566818s delta_ub -1.980766416s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a000 con 0x3752400 2026-02-11T23:30:31.568+0000 7fac71bfd640 20 osd.0 66 get_con_osd_cluster to osd.1 from_epoch 66 2026-02-11T23:30:31.568+0000 7fac71bfd640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:31.568+0000 7fac71bfd640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:31.568+0000 7fac71bfd640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 66 is not newer than session's projected_epoch of 68 2026-02-11T23:30:31.568+0000 7fac71bfd640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980916023s,-1.980766416s]) 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 get_con_osd_cluster to osd.2 from_epoch 66 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:31.568+0000 7fac723fe640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 65 to ping map epoch of 68 2026-02-11T23:30:31.568+0000 7fac723fe640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 66 is not newer than session's projected_epoch of 68 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649102211s,-4.648936749s]) 2026-02-11T23:30:31.568+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 19 ==== osd_map(67..68 src has 1..68) ==== 878+0+0 (unknown 0 0 0) 0x46ea8c0 con 0x3752000 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 OSD::ms_dispatch: osd_map(67..68 src has 1..68) 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 _dispatch 0x46ea8c0 osd_map(67..68 src has 1..68) 2026-02-11T23:30:31.568+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 4 ==== osd_ping(ping_reply e68 up_from 21 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 14.517558098s delta_ub -9.886775017s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bc80 con 0x3740c00 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 get_con_osd_cluster to osd.4 from_epoch 66 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:31.568+0000 7fac703fa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 66 to ping map epoch of 68 2026-02-11T23:30:31.568+0000 7fac703fa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 66 is not newer than session's projected_epoch of 68 2026-02-11T23:30:31.568+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 6 ==== osd_ping(ping_reply e68 up_from 13 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 19.755428314s delta_ub -4.648905754s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a000 con 0x34c1000 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886958122s,-9.886775017s]) 2026-02-11T23:30:31.568+0000 7fac71bfd640 20 osd.0 66 get_con_osd_cluster to osd.2 from_epoch 66 2026-02-11T23:30:31.568+0000 7fac71bfd640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:31.568+0000 7fac71bfd640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:31.568+0000 7fac71bfd640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 66 is not newer than session's projected_epoch of 68 2026-02-11T23:30:31.568+0000 7fac71bfd640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649096012s,-4.648905754s]) 2026-02-11T23:30:31.568+0000 7fac606e4640 3 osd.0 66 handle_osd_map epochs [67,68], i have 66, src has [1,68] 2026-02-11T23:30:31.568+0000 7fac606e4640 10 osd.0 66 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:31.568+0000 7fac606e4640 10 osd.0 66 handle_osd_map got inc map for epoch 67 2026-02-11T23:30:31.568+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 4 ==== osd_ping(ping_reply e68 up_from 21 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 14.517588615s delta_ub -9.886744499s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x3740800 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 get_con_osd_cluster to osd.4 from_epoch 66 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:31.568+0000 7fac723fe640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 66 is not newer than session's projected_epoch of 68 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886969566s,-9.886744499s]) 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 got_full_map 67, nothing requested 2026-02-11T23:30:31.568+0000 7fac606e4640 10 osd.0 66 handle_osd_map got inc map for epoch 68 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 got_full_map 68, nothing requested 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:31.568+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 4 ==== osd_ping(ping_reply e68 up_from 17 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 17.094053268s delta_ub -7.310281277s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bc80 con 0x3740000 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 get_con_osd_cluster to osd.3 from_epoch 66 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:31.568+0000 7fac703fa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 66 to ping map epoch of 68 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 handle_osd_map pg_num_history pg_num_history(e68 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1}} deleted_pools ) 2026-02-11T23:30:31.568+0000 7fac703fa640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 66 is not newer than session's projected_epoch of 68 2026-02-11T23:30:31.568+0000 7fac703fa640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310572147s,-7.310281277s]) 2026-02-11T23:30:31.568+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {67={},68={}} 2026-02-11T23:30:31.568+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:31.568+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e68 maps [1~68] lci=[5,68] tlb=1) 2026-02-11T23:30:31.568+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 4 ==== osd_ping(ping_reply e68 up_from 17 ping_stamp 2026-02-11T23:30:31.569918+0000/24.404333115s send_stamp 17.094223022s delta_ub -7.310110092s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x3740400 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 get_con_osd_cluster to osd.3 from_epoch 66 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 get_nextmap_reserved map_reservations: {66=1} 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 release_map epoch: 66 2026-02-11T23:30:31.568+0000 7fac723fe640 10 osd.0 66 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 66 is not newer than session's projected_epoch of 68 2026-02-11T23:30:31.568+0000 7fac723fe640 20 osd.0 66 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310451508s,-7.310110092s]) 2026-02-11T23:30:31.568+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.2 v2:127.0.0.1:6820/2662569638 4 ==== osd_map(67..68 src has 1..68) ==== 878+0+0 (unknown 0 0 0) 0x46eafc0 con 0x3f57c00 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 OSD::ms_dispatch: osd_map(67..68 src has 1..68) 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 _dispatch 0x46eafc0 osd_map(67..68 src has 1..68) 2026-02-11T23:30:31.568+0000 7fac606e4640 3 osd.0 66 handle_osd_map epochs [67,68], i have 68, src has [1,68] 2026-02-11T23:30:31.568+0000 7fac606e4640 10 osd.0 66 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:31.568+0000 7fac606e4640 10 osd.0 66 no new maps here, dropping 2026-02-11T23:30:31.568+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 21 ==== osd_map(67..68 src has 1..68) ==== 878+0+0 (unknown 0 0 0) 0x46eac40 con 0x3741000 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 OSD::ms_dispatch: osd_map(67..68 src has 1..68) 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 _dispatch 0x46eac40 osd_map(67..68 src has 1..68) 2026-02-11T23:30:31.568+0000 7fac606e4640 3 osd.0 66 handle_osd_map epochs [67,68], i have 68, src has [1,68] 2026-02-11T23:30:31.568+0000 7fac606e4640 10 osd.0 66 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:31.568+0000 7fac606e4640 10 osd.0 66 no new maps here, dropping 2026-02-11T23:30:31.568+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 27 ==== osd_map(67..68 src has 1..68) ==== 878+0+0 (unknown 0 0 0) 0x38e3880 con 0x3e4a000 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 OSD::ms_dispatch: osd_map(67..68 src has 1..68) 2026-02-11T23:30:31.568+0000 7fac606e4640 20 osd.0 66 _dispatch 0x38e3880 osd_map(67..68 src has 1..68) 2026-02-11T23:30:31.569+0000 7fac606e4640 3 osd.0 66 handle_osd_map epochs [67,68], i have 68, src has [1,68] 2026-02-11T23:30:31.569+0000 7fac606e4640 10 osd.0 66 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:31.569+0000 7fac606e4640 10 osd.0 66 no new maps here, dropping 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0 66 _committed_osd_maps 67..68 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0 66 advance to epoch 67 (<= last 68 <= newest_map 68) 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0 67 advance to epoch 68 (<= last 68 <= newest_map 68) 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0 68 consume_map version 68 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0 66 await_reserved_maps epoch:68 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0 66 await_reserved_maps done 0 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0 68 consume_map 68 -- publish done 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0 68 identify_splits_and_merges 4.0 e66 to e68 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0 68 identify_splits_and_merges 4.0 e66 to e68 pg_nums {45=1} 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0 68 identify_splits_and_merges 1.0 e66 to e68 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0 68 identify_splits_and_merges 1.0 e66 to e68 pg_nums {24=1} 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0 68 prune_pg_created 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0:0.consume_map 68 (was 66) 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0:1.consume_map 68 (was 66) 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0:2.consume_map 68 (was 66) 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0:3.consume_map 68 (was 66) 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0:4.consume_map 68 (was 66) 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0:5.consume_map 68 (was 66) 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0:6.consume_map 68 (was 66) 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0:7.consume_map 68 (was 66) 2026-02-11T23:30:31.569+0000 7fac5e983640 10 osd.0 68 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:31.569+0000 7fac5e983640 15 osd.0 68 enqueue_peering_evt 4.0 epoch_sent: 68 epoch_requested: 68 NullEvt 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2 prio 255 cost 10 e68) 2026-02-11T23:30:31.569+0000 7fac5e983640 15 osd.0 68 enqueue_peering_evt 1.0 epoch_sent: 68 epoch_requested: 68 NullEvt 2026-02-11T23:30:31.569+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2 prio 255 cost 10 e68) 2026-02-11T23:30:31.569+0000 7fac5e983640 7 osd.0 68 activate_map version 68 2026-02-11T23:30:31.569+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.569+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2immediate prio 255 cost 10 e68) queued 2026-02-11T23:30:31.569+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.569+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2immediate prio 255 cost 10 e68) pg 0x4856000 2026-02-11T23:30:31.569+0000 7fac55971640 20 osd.0 68 agent_entry empty queue 2026-02-11T23:30:31.569+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.569+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2immediate prio 255 cost 10 e68) queued 2026-02-11T23:30:31.569+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.569+0000 7fac51168640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 67 2026-02-11T23:30:31.569+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 68 epoch_requested: 68 NullEvt) class_id 2immediate prio 255 cost 10 e68) pg 0x40f8000 2026-02-11T23:30:31.569+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 66 -> 67 2026-02-11T23:30:31.569+0000 7fac55170640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 67 2026-02-11T23:30:31.569+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 66 -> 67 2026-02-11T23:30:31.569+0000 7fac51168640 10 osd.0 pg_epoch: 66 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:31.569+0000 7fac55170640 10 osd.0 pg_epoch: 66 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:31.569+0000 7fac51168640 10 osd.0 pg_epoch: 67 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:31.569+0000 7fac55170640 10 osd.0 pg_epoch: 67 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:31.569+0000 7fac51168640 10 osd.0 pg_epoch: 67 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:31.569+0000 7fac55170640 10 osd.0 pg_epoch: 67 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:31.569+0000 7fac51168640 20 osd.0 68 advance_pg new pool opts old pool opts 2026-02-11T23:30:31.569+0000 7fac55170640 20 osd.0 68 advance_pg new pool opts old pool opts 2026-02-11T23:30:31.569+0000 7fac55170640 10 osd.0 pg_epoch: 67 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 68 2026-02-11T23:30:31.569+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 67 -> 68 2026-02-11T23:30:31.569+0000 7fac51168640 10 osd.0 pg_epoch: 67 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 68 2026-02-11T23:30:31.569+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 67 -> 68 2026-02-11T23:30:31.569+0000 7fac55170640 10 osd.0 pg_epoch: 67 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:31.569+0000 7fac51168640 10 osd.0 pg_epoch: 67 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:31.569+0000 7fac55170640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:31.569+0000 7fac51168640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:31.569+0000 7fac55170640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:31.569+0000 7fac51168640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:31.569+0000 7fac55170640 20 osd.0 68 advance_pg new pool opts old pool opts 2026-02-11T23:30:31.569+0000 7fac51168640 20 osd.0 68 advance_pg new pool opts old pool opts 2026-02-11T23:30:31.569+0000 7fac51168640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 67..68 2026-02-11T23:30:31.569+0000 7fac55170640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 67..68 2026-02-11T23:30:31.569+0000 7fac51168640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:31.569+0000 7fac55170640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:31.569+0000 7fac51168640 20 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 68 2026-02-11T23:30:31.569+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:31.569+0000 7fac55170640 20 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 28 while current is 68 2026-02-11T23:30:31.569+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:31.569+0000 7fac51168640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 68 epoch_requested: 68 NullEvt 2026-02-11T23:30:31.569+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:31.569+0000 7fac55170640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 68 epoch_requested: 68 NullEvt 2026-02-11T23:30:31.569+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:31.569+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.569+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.782+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 42 ==== osd_map(69..70 src has 1..70) ==== 1208+0+0 (crc 0 0 0) 0x46ea540 con 0x396bc00 2026-02-11T23:30:31.782+0000 7fac626e8640 20 osd.0 68 OSD::ms_dispatch: osd_map(69..70 src has 1..70) 2026-02-11T23:30:31.782+0000 7fac626e8640 20 osd.0 68 _dispatch 0x46ea540 osd_map(69..70 src has 1..70) 2026-02-11T23:30:31.782+0000 7fac626e8640 3 osd.0 68 handle_osd_map epochs [69,70], i have 68, src has [1,70] 2026-02-11T23:30:31.782+0000 7fac626e8640 10 osd.0 68 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:31.782+0000 7fac626e8640 10 osd.0 68 handle_osd_map got inc map for epoch 69 2026-02-11T23:30:31.782+0000 7fac626e8640 20 osd.0 68 got_full_map 69, nothing requested 2026-02-11T23:30:31.782+0000 7fac626e8640 10 osd.0 68 handle_osd_map got inc map for epoch 70 2026-02-11T23:30:31.782+0000 7fac626e8640 20 osd.0 68 got_full_map 70, nothing requested 2026-02-11T23:30:31.782+0000 7fac626e8640 20 osd.0 68 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:31.782+0000 7fac626e8640 20 osd.0 68 handle_osd_map pg_num_history pg_num_history(e70 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1}} deleted_pools ) 2026-02-11T23:30:31.782+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {69={},70={}} 2026-02-11T23:30:31.782+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:31.782+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e70 maps [1~70] lci=[5,70] tlb=1) 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0 68 _committed_osd_maps 69..70 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0 68 advance to epoch 69 (<= last 70 <= newest_map 70) 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0 69 advance to epoch 70 (<= last 70 <= newest_map 70) 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0 70 consume_map version 70 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0 68 await_reserved_maps epoch:70 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0 68 await_reserved_maps done 0 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0 70 consume_map 70 -- publish done 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0 70 identify_splits_and_merges 4.0 e68 to e70 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0 70 identify_splits_and_merges 4.0 e68 to e70 pg_nums {45=1} 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0 70 identify_splits_and_merges 1.0 e68 to e70 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0 70 identify_splits_and_merges 1.0 e68 to e70 pg_nums {24=1} 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0 70 prune_pg_created 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0:0.consume_map 70 (was 68) 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0:1.consume_map 70 (was 68) 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0:2.consume_map 70 (was 68) 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0:3.consume_map 70 (was 68) 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0:4.consume_map 70 (was 68) 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0:5.consume_map 70 (was 68) 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0:6.consume_map 70 (was 68) 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0:7.consume_map 70 (was 68) 2026-02-11T23:30:31.782+0000 7fac5e983640 10 osd.0 70 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:31.782+0000 7fac5e983640 15 osd.0 70 enqueue_peering_evt 4.0 epoch_sent: 70 epoch_requested: 70 NullEvt 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2 prio 255 cost 10 e70) 2026-02-11T23:30:31.782+0000 7fac5e983640 15 osd.0 70 enqueue_peering_evt 1.0 epoch_sent: 70 epoch_requested: 70 NullEvt 2026-02-11T23:30:31.782+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2 prio 255 cost 10 e70) 2026-02-11T23:30:31.782+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.782+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2immediate prio 255 cost 10 e70) queued 2026-02-11T23:30:31.782+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.782+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2immediate prio 255 cost 10 e70) pg 0x4856000 2026-02-11T23:30:31.782+0000 7fac5e983640 7 osd.0 70 activate_map version 70 2026-02-11T23:30:31.782+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:31.782+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2immediate prio 255 cost 10 e70) queued 2026-02-11T23:30:31.782+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:31.782+0000 7fac51168640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 69 2026-02-11T23:30:31.782+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 70 epoch_requested: 70 NullEvt) class_id 2immediate prio 255 cost 10 e70) pg 0x40f8000 2026-02-11T23:30:31.782+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 68 -> 69 2026-02-11T23:30:31.782+0000 7fac55971640 20 osd.0 70 agent_entry empty queue 2026-02-11T23:30:31.782+0000 7fac51168640 10 osd.0 pg_epoch: 68 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:31.782+0000 7fac55170640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 69 2026-02-11T23:30:31.782+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 68 -> 69 2026-02-11T23:30:31.782+0000 7fac51168640 10 osd.0 pg_epoch: 69 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:31.782+0000 7fac55170640 10 osd.0 pg_epoch: 68 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:31.782+0000 7fac51168640 10 osd.0 pg_epoch: 69 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:31.782+0000 7fac55170640 10 osd.0 pg_epoch: 69 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:31.782+0000 7fac51168640 20 osd.0 70 advance_pg new pool opts old pool opts 2026-02-11T23:30:31.782+0000 7fac55170640 10 osd.0 pg_epoch: 69 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:31.782+0000 7fac55170640 20 osd.0 70 advance_pg new pool opts old pool opts 2026-02-11T23:30:31.782+0000 7fac51168640 10 osd.0 pg_epoch: 69 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 70 2026-02-11T23:30:31.782+0000 7fac55170640 10 osd.0 pg_epoch: 69 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 70 2026-02-11T23:30:31.782+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 69 -> 70 2026-02-11T23:30:31.782+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 69 -> 70 2026-02-11T23:30:31.782+0000 7fac55170640 10 osd.0 pg_epoch: 69 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:31.782+0000 7fac51168640 10 osd.0 pg_epoch: 69 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:31.782+0000 7fac55170640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:31.782+0000 7fac55170640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:31.782+0000 7fac51168640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:31.782+0000 7fac55170640 20 osd.0 70 advance_pg new pool opts old pool opts 2026-02-11T23:30:31.782+0000 7fac55170640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 69..70 2026-02-11T23:30:31.782+0000 7fac51168640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:31.782+0000 7fac55170640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:31.782+0000 7fac51168640 20 osd.0 70 advance_pg new pool opts old pool opts 2026-02-11T23:30:31.782+0000 7fac55170640 20 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Dirtying info: last_persisted is 28 while current is 70 2026-02-11T23:30:31.782+0000 7fac51168640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 69..70 2026-02-11T23:30:31.782+0000 7fac51168640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:31.782+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:31.782+0000 7fac51168640 20 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 70 2026-02-11T23:30:31.782+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:31.782+0000 7fac51168640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 70 epoch_requested: 70 NullEvt 2026-02-11T23:30:31.782+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:31.782+0000 7fac55170640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 70 epoch_requested: 70 NullEvt 2026-02-11T23:30:31.782+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:31.782+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.782+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.782+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.782+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:31.978+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 5 ==== osd_ping(ping e68 up_from 17 ping_stamp 2026-02-11T23:30:31.979946+0000/17.503984451s send_stamp 17.503984451s delta_ub -7.310110092s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a000 con 0x3f56800 2026-02-11T23:30:31.978+0000 7fac71bfd640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310458183s,-7.310110092s]) 2026-02-11T23:30:31.978+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e70 up_from 5 ping_stamp 2026-02-11T23:30:31.979946+0000/17.503984451s send_stamp 24.814439774s delta_ub 7.310458183s) -- 0x3b1ad80 con 0x3f56800 2026-02-11T23:30:31.978+0000 7fac71bfd640 20 osd.0 70 get_con_osd_cluster to osd.3 from_epoch 70 2026-02-11T23:30:31.978+0000 7fac71bfd640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:31.978+0000 7fac71bfd640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:31.978+0000 7fac71bfd640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6828/1114810440 map epoch 68 -> 70 (shared) 2026-02-11T23:30:31.978+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 5 ==== osd_ping(ping e68 up_from 17 ping_stamp 2026-02-11T23:30:31.979946+0000/17.503984451s send_stamp 17.503984451s delta_ub -7.310110092s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x3f56c00 2026-02-11T23:30:31.978+0000 7fac71bfd640 10 osd.0 70 send_incremental_map epoch range: (69, 70] to v2:127.0.0.1:6828/1114810440 2026-02-11T23:30:31.978+0000 7fac71bfd640 10 osd.0 70 add_map_inc_bl 69 582 bytes 2026-02-11T23:30:31.978+0000 7fac71bfd640 10 osd.0 70 add_map_inc_bl 70 574 bytes 2026-02-11T23:30:31.978+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- osd_map(69..70 src has 1..70) -- 0x46ea700 con 0x3741000 2026-02-11T23:30:31.978+0000 7fac723fe640 20 osd.0 70 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310756683s,-7.310110092s]) 2026-02-11T23:30:31.978+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e70 up_from 5 ping_stamp 2026-02-11T23:30:31.979946+0000/17.503984451s send_stamp 24.814739227s delta_ub 7.310756683s) -- 0x3b1bb00 con 0x3f56c00 2026-02-11T23:30:31.978+0000 7fac723fe640 20 osd.0 70 get_con_osd_cluster to osd.3 from_epoch 70 2026-02-11T23:30:31.978+0000 7fac723fe640 20 osd.0 70 get_nextmap_reserved map_reservations: {70=1} 2026-02-11T23:30:31.978+0000 7fac723fe640 20 osd.0 70 release_map epoch: 70 2026-02-11T23:30:31.978+0000 7fac723fe640 10 osd.0 70 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 70 is not newer than session's projected_epoch of 70 2026-02-11T23:30:32.051+0000 7fac68beb640 10 osd.0 70 tick 2026-02-11T23:30:32.051+0000 7fac68beb640 20 osd.0 70 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:32.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:32.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:32.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:02.168735+0000) 2026-02-11T23:30:32.370+0000 7fac683ea640 10 osd.0 70 tick_without_osd_lock 2026-02-11T23:30:32.370+0000 7fac683ea640 0 osd.0 70 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:32.370+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:32.370+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:32.370+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:32.370+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.140 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:32.370+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:32.371, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:32.370+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:32.370+0000 7fac683ea640 10 osd.0 70 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:32.370+0000 7fac683ea640 20 osd.0 70 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:32.370+0000 7fac683ea640 10 osd.0 70 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:32.370+0000 7fac683ea640 20 osd.0 70 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:33.009+0000 7fac68beb640 10 osd.0 70 tick 2026-02-11T23:30:33.009+0000 7fac68beb640 20 osd.0 70 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:33.010+0000 7fac60ee5640 20 osd.0 70 reports for 0 queries 2026-02-11T23:30:33.010+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x34c3500 con 0x396a800 2026-02-11T23:30:33.010+0000 7fac60ee5640 15 osd.0 70 collect_pg_stats 2026-02-11T23:30:33.010+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836486 v 0) -- 0x489d800 con 0x396a800 2026-02-11T23:30:33.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:33.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:33.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:03.168799+0000) 2026-02-11T23:30:33.329+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 43 ==== osd_map(75..76 src has 1..76) ==== 1200+0+0 (crc 0 0 0) 0x46ea700 con 0x396bc00 2026-02-11T23:30:33.329+0000 7fac626e8640 20 osd.0 70 OSD::ms_dispatch: osd_map(75..76 src has 1..76) 2026-02-11T23:30:33.329+0000 7fac626e8640 20 osd.0 70 _dispatch 0x46ea700 osd_map(75..76 src has 1..76) 2026-02-11T23:30:33.329+0000 7fac626e8640 10 monclient: _renew_subs 2026-02-11T23:30:33.329+0000 7fac626e8640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:33.329+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=71}) -- 0x449da00 con 0x396bc00 2026-02-11T23:30:33.329+0000 7fac626e8640 3 osd.0 70 handle_osd_map epochs [75,76], i have 70, src has [1,76] 2026-02-11T23:30:33.329+0000 7fac626e8640 10 osd.0 70 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:33.329+0000 7fac626e8640 10 osd.0 70 handle_osd_map message skips epochs 71..74 2026-02-11T23:30:33.330+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 44 ==== osd_map(71..76 src has 1..76) ==== 3223+0+0 (crc 0 0 0) 0x46ea1c0 con 0x396bc00 2026-02-11T23:30:33.330+0000 7fac626e8640 20 osd.0 70 OSD::ms_dispatch: osd_map(71..76 src has 1..76) 2026-02-11T23:30:33.330+0000 7fac626e8640 20 osd.0 70 _dispatch 0x46ea1c0 osd_map(71..76 src has 1..76) 2026-02-11T23:30:33.330+0000 7fac626e8640 3 osd.0 70 handle_osd_map epochs [71,76], i have 70, src has [1,76] 2026-02-11T23:30:33.330+0000 7fac626e8640 10 osd.0 70 handle_osd_map: received 0 full maps and 6 incremental maps 2026-02-11T23:30:33.330+0000 7fac626e8640 10 osd.0 70 handle_osd_map got inc map for epoch 71 2026-02-11T23:30:33.330+0000 7fac626e8640 20 osd.0 70 got_full_map 71, nothing requested 2026-02-11T23:30:33.330+0000 7fac626e8640 10 osd.0 70 handle_osd_map got inc map for epoch 72 2026-02-11T23:30:33.330+0000 7fac626e8640 20 osd.0 70 got_full_map 72, nothing requested 2026-02-11T23:30:33.330+0000 7fac626e8640 10 osd.0 70 handle_osd_map got inc map for epoch 73 2026-02-11T23:30:33.330+0000 7fac626e8640 20 osd.0 70 got_full_map 73, nothing requested 2026-02-11T23:30:33.330+0000 7fac626e8640 10 osd.0 70 handle_osd_map got inc map for epoch 74 2026-02-11T23:30:33.330+0000 7fac626e8640 20 osd.0 70 got_full_map 74, nothing requested 2026-02-11T23:30:33.330+0000 7fac626e8640 10 osd.0 70 handle_osd_map got inc map for epoch 75 2026-02-11T23:30:33.330+0000 7fac626e8640 20 osd.0 70 got_full_map 75, nothing requested 2026-02-11T23:30:33.330+0000 7fac626e8640 10 osd.0 70 handle_osd_map got inc map for epoch 76 2026-02-11T23:30:33.330+0000 7fac626e8640 20 osd.0 70 got_full_map 76, nothing requested 2026-02-11T23:30:33.330+0000 7fac626e8640 10 osd.0 70 _track_pools_and_pg_num_changes recording new pool 8 pg_num 1 2026-02-11T23:30:33.330+0000 7fac626e8640 20 osd.0 70 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:33.330+0000 7fac626e8640 20 osd.0 70 handle_osd_map pg_num_history pg_num_history(e76 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1}} deleted_pools ) 2026-02-11T23:30:33.330+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {71={},72={},73={},74={},75={},76={}} 2026-02-11T23:30:33.330+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:33.330+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e76 maps [1~76] lci=[5,76] tlb=1) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0 70 _committed_osd_maps 71..76 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0 70 advance to epoch 71 (<= last 76 <= newest_map 76) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0 71 advance to epoch 72 (<= last 76 <= newest_map 76) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0 72 advance to epoch 73 (<= last 76 <= newest_map 76) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0 73 advance to epoch 74 (<= last 76 <= newest_map 76) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0 74 advance to epoch 75 (<= last 76 <= newest_map 76) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0 75 advance to epoch 76 (<= last 76 <= newest_map 76) 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0 76 consume_map version 76 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0 70 await_reserved_maps epoch:76 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0 70 await_reserved_maps done 0 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0 76 consume_map 76 -- publish done 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0 76 identify_splits_and_merges 4.0 e70 to e76 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0 76 identify_splits_and_merges 4.0 e70 to e76 pg_nums {45=1} 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0 76 identify_splits_and_merges 1.0 e70 to e76 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0 76 identify_splits_and_merges 1.0 e70 to e76 pg_nums {24=1} 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0 76 prune_pg_created 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0:0.consume_map 76 (was 70) 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0:1.consume_map 76 (was 70) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0:2.consume_map 76 (was 70) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0:3.consume_map 76 (was 70) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0:4.consume_map 76 (was 70) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0:5.consume_map 76 (was 70) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0:6.consume_map 76 (was 70) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0:7.consume_map 76 (was 70) 2026-02-11T23:30:33.331+0000 7fac5e983640 10 osd.0 76 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:33.331+0000 7fac5e983640 15 osd.0 76 enqueue_peering_evt 4.0 epoch_sent: 76 epoch_requested: 76 NullEvt 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2 prio 255 cost 10 e76) 2026-02-11T23:30:33.331+0000 7fac5e983640 15 osd.0 76 enqueue_peering_evt 1.0 epoch_sent: 76 epoch_requested: 76 NullEvt 2026-02-11T23:30:33.331+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2 prio 255 cost 10 e76) 2026-02-11T23:30:33.331+0000 7fac5e983640 7 osd.0 76 activate_map version 76 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2immediate prio 255 cost 10 e76) queued 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2immediate prio 255 cost 10 e76) pg 0x4856000 2026-02-11T23:30:33.331+0000 7fac55971640 20 osd.0 76 agent_entry empty queue 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2immediate prio 255 cost 10 e76) queued 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 71 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 76 epoch_requested: 76 NullEvt) class_id 2immediate prio 255 cost 10 e76) pg 0x40f8000 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 70 -> 71 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 70 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 71 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 70 -> 71 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 70 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 71 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 71 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 71 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 71 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 71 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 72 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 71 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 72 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 71 -> 72 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 71 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 71 -> 72 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 72 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 71 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 72 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 72 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 72 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 72 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 73 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 72 -> 73 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 72 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 73 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 72 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 72 -> 73 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 72 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 74 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 74 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 73 -> 74 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 73 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 73 -> 74 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 73 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 75 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 75 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 74 -> 75 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 74 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 74 -> 75 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 75 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 74 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 75 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 75 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 75 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 75 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 76 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 75 -> 76 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 75 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 75 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 76 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 75 -> 76 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 75 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 71..76 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 76 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 76 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.331+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 71..76 2026-02-11T23:30:33.331+0000 7fac51168640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 76 epoch_requested: 76 NullEvt 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:33.331+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 76 2026-02-11T23:30:33.331+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:33.331+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.331+0000 7fac55170640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 76 epoch_requested: 76 NullEvt 2026-02-11T23:30:33.331+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:33.331+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.343+0000 7fac683ea640 10 osd.0 76 tick_without_osd_lock 2026-02-11T23:30:33.343+0000 7fac683ea640 0 osd.0 76 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:33.343+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:33.343+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:33.343+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:33.343+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.140 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:33.343+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:33.345, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:33.343+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:33.343+0000 7fac683ea640 10 osd.0 76 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:33.343+0000 7fac683ea640 20 osd.0 76 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:33.343+0000 7fac683ea640 10 osd.0 76 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:33.343+0000 7fac683ea640 20 osd.0 76 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:33.633+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 45 ==== osd_map(77..78 src has 1..78) ==== 1200+0+0 (crc 0 0 0) 0x46ea000 con 0x396bc00 2026-02-11T23:30:33.633+0000 7fac626e8640 20 osd.0 76 OSD::ms_dispatch: osd_map(77..78 src has 1..78) 2026-02-11T23:30:33.633+0000 7fac626e8640 20 osd.0 76 _dispatch 0x46ea000 osd_map(77..78 src has 1..78) 2026-02-11T23:30:33.633+0000 7fac626e8640 3 osd.0 76 handle_osd_map epochs [77,78], i have 76, src has [1,78] 2026-02-11T23:30:33.633+0000 7fac626e8640 10 osd.0 76 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:33.633+0000 7fac626e8640 10 osd.0 76 handle_osd_map got inc map for epoch 77 2026-02-11T23:30:33.633+0000 7fac626e8640 20 osd.0 76 got_full_map 77, nothing requested 2026-02-11T23:30:33.633+0000 7fac626e8640 10 osd.0 76 handle_osd_map got inc map for epoch 78 2026-02-11T23:30:33.633+0000 7fac626e8640 20 osd.0 76 got_full_map 78, nothing requested 2026-02-11T23:30:33.633+0000 7fac626e8640 20 osd.0 76 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:33.633+0000 7fac626e8640 20 osd.0 76 handle_osd_map pg_num_history pg_num_history(e78 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1}} deleted_pools ) 2026-02-11T23:30:33.633+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {77={},78={}} 2026-02-11T23:30:33.633+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:33.633+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e78 maps [1~78] lci=[5,78] tlb=1) 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0 76 _committed_osd_maps 77..78 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0 76 advance to epoch 77 (<= last 78 <= newest_map 78) 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0 77 advance to epoch 78 (<= last 78 <= newest_map 78) 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0 78 consume_map version 78 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0 76 await_reserved_maps epoch:78 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0 76 await_reserved_maps done 0 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0 78 consume_map 78 -- publish done 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0 78 identify_splits_and_merges 4.0 e76 to e78 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0 78 identify_splits_and_merges 4.0 e76 to e78 pg_nums {45=1} 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0 78 identify_splits_and_merges 1.0 e76 to e78 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0 78 identify_splits_and_merges 1.0 e76 to e78 pg_nums {24=1} 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0 78 prune_pg_created 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0:0.consume_map 78 (was 76) 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0:1.consume_map 78 (was 76) 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0:2.consume_map 78 (was 76) 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0:3.consume_map 78 (was 76) 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0:4.consume_map 78 (was 76) 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0:5.consume_map 78 (was 76) 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0:6.consume_map 78 (was 76) 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0:7.consume_map 78 (was 76) 2026-02-11T23:30:33.634+0000 7fac5e983640 10 osd.0 78 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:33.634+0000 7fac5e983640 15 osd.0 78 enqueue_peering_evt 4.0 epoch_sent: 78 epoch_requested: 78 NullEvt 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2 prio 255 cost 10 e78) 2026-02-11T23:30:33.634+0000 7fac5e983640 15 osd.0 78 enqueue_peering_evt 1.0 epoch_sent: 78 epoch_requested: 78 NullEvt 2026-02-11T23:30:33.634+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2 prio 255 cost 10 e78) 2026-02-11T23:30:33.634+0000 7fac5e983640 7 osd.0 78 activate_map version 78 2026-02-11T23:30:33.634+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:33.634+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2immediate prio 255 cost 10 e78) queued 2026-02-11T23:30:33.634+0000 7fac55971640 20 osd.0 78 agent_entry empty queue 2026-02-11T23:30:33.634+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:33.634+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2immediate prio 255 cost 10 e78) pg 0x4856000 2026-02-11T23:30:33.634+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:33.634+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2immediate prio 255 cost 10 e78) queued 2026-02-11T23:30:33.634+0000 7fac51168640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 77 2026-02-11T23:30:33.634+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:33.634+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 78 epoch_requested: 78 NullEvt) class_id 2immediate prio 255 cost 10 e78) pg 0x40f8000 2026-02-11T23:30:33.634+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 76 -> 77 2026-02-11T23:30:33.634+0000 7fac51168640 10 osd.0 pg_epoch: 76 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:33.634+0000 7fac51168640 10 osd.0 pg_epoch: 77 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.634+0000 7fac55170640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 77 2026-02-11T23:30:33.634+0000 7fac51168640 10 osd.0 pg_epoch: 77 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.634+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 76 -> 77 2026-02-11T23:30:33.634+0000 7fac51168640 20 osd.0 78 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.634+0000 7fac55170640 10 osd.0 pg_epoch: 76 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.634+0000 7fac51168640 10 osd.0 pg_epoch: 77 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 78 2026-02-11T23:30:33.634+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 77 -> 78 2026-02-11T23:30:33.634+0000 7fac55170640 10 osd.0 pg_epoch: 77 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.634+0000 7fac51168640 10 osd.0 pg_epoch: 77 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:33.634+0000 7fac55170640 10 osd.0 pg_epoch: 77 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.634+0000 7fac51168640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.634+0000 7fac51168640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.634+0000 7fac55170640 20 osd.0 78 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.634+0000 7fac51168640 20 osd.0 78 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.634+0000 7fac51168640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 77..78 2026-02-11T23:30:33.634+0000 7fac51168640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:33.634+0000 7fac55170640 10 osd.0 pg_epoch: 77 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 78 2026-02-11T23:30:33.634+0000 7fac51168640 20 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 78 2026-02-11T23:30:33.634+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 77 -> 78 2026-02-11T23:30:33.634+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:33.634+0000 7fac55170640 10 osd.0 pg_epoch: 77 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:33.634+0000 7fac51168640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 78 epoch_requested: 78 NullEvt 2026-02-11T23:30:33.634+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:33.634+0000 7fac55170640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:33.634+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:33.634+0000 7fac55170640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:33.634+0000 7fac55170640 20 osd.0 78 advance_pg new pool opts old pool opts 2026-02-11T23:30:33.634+0000 7fac55170640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 77..78 2026-02-11T23:30:33.634+0000 7fac55170640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:33.634+0000 7fac55170640 20 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 78 2026-02-11T23:30:33.634+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:33.634+0000 7fac55170640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 78 epoch_requested: 78 NullEvt 2026-02-11T23:30:33.634+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:33.634+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.034+0000 7fac68beb640 10 osd.0 78 tick 2026-02-11T23:30:34.034+0000 7fac68beb640 20 osd.0 78 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:34.156+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 28 ==== pg_lease(4.0 pg_lease(ru 25.104763031s ub 33.104816437s int 16.000000000s) e82/82) ==== 58+0+0 (unknown 0 0 0) 0x3b1bc80 con 0x3e4a000 2026-02-11T23:30:34.156+0000 7fac703fa640 15 osd.0 78 enqueue_peering_evt 4.0 epoch_sent: 82 epoch_requested: 82 MLease epoch 82 from osd.4 pg_lease(ru 25.104763031s ub 33.104816437s int 16.000000000s) 2026-02-11T23:30:34.156+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 MLease epoch 82 from osd.4 pg_lease(ru 25.104763031s ub 33.104816437s int 16.000000000s)) class_id 2 prio 255 cost 10 e82) 2026-02-11T23:30:34.156+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:34.156+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 MLease epoch 82 from osd.4 pg_lease(ru 25.104763031s ub 33.104816437s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e82) queued 2026-02-11T23:30:34.156+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:34.156+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 MLease epoch 82 from osd.4 pg_lease(ru 25.104763031s ub 33.104816437s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e82) pg 0x4856000 2026-02-11T23:30:34.156+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 82, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 MLease epoch 82 from osd.4 pg_lease(ru 25.104763031s ub 33.104816437s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e82) 2026-02-11T23:30:34.156+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.156+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:34.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:34.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:04.168864+0000) 2026-02-11T23:30:34.279+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 6 ==== osd_ping(ping e81 up_from 17 ping_stamp 2026-02-11T23:30:34.280140+0000/19.804176331s send_stamp 19.804176331s delta_ub -7.310319901s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ad80 con 0x3f56800 2026-02-11T23:30:34.279+0000 7fac71bfd640 20 osd.0 78 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310447216s,-7.310319901s]) 2026-02-11T23:30:34.279+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e78 up_from 5 ping_stamp 2026-02-11T23:30:34.280140+0000/19.804176331s send_stamp 27.114622116s delta_ub 7.310447216s) -- 0x3b1a000 con 0x3f56800 2026-02-11T23:30:34.279+0000 7fac71bfd640 20 osd.0 78 get_con_osd_cluster to osd.3 from_epoch 78 2026-02-11T23:30:34.279+0000 7fac71bfd640 20 osd.0 78 get_nextmap_reserved map_reservations: {78=1} 2026-02-11T23:30:34.279+0000 7fac71bfd640 20 osd.0 78 release_map epoch: 78 2026-02-11T23:30:34.279+0000 7fac71bfd640 10 osd.0 78 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 70 to ping map epoch of 81 2026-02-11T23:30:34.279+0000 7fac71bfd640 10 osd.0 78 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 78 is not newer than session's projected_epoch of 81 2026-02-11T23:30:34.279+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 6 ==== osd_ping(ping e81 up_from 17 ping_stamp 2026-02-11T23:30:34.280140+0000/19.804176331s send_stamp 19.804176331s delta_ub -7.310319901s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bb00 con 0x3f56c00 2026-02-11T23:30:34.279+0000 7fac723fe640 20 osd.0 78 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310496330s,-7.310319901s]) 2026-02-11T23:30:34.279+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e78 up_from 5 ping_stamp 2026-02-11T23:30:34.280140+0000/19.804176331s send_stamp 27.114671707s delta_ub 7.310496330s) -- 0x3b1be00 con 0x3f56c00 2026-02-11T23:30:34.279+0000 7fac723fe640 20 osd.0 78 get_con_osd_cluster to osd.3 from_epoch 78 2026-02-11T23:30:34.279+0000 7fac723fe640 20 osd.0 78 get_nextmap_reserved map_reservations: {78=1} 2026-02-11T23:30:34.279+0000 7fac723fe640 20 osd.0 78 release_map epoch: 78 2026-02-11T23:30:34.279+0000 7fac723fe640 10 osd.0 78 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 78 is not newer than session's projected_epoch of 81 2026-02-11T23:30:34.279+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 22 ==== osd_map(79..81 src has 1..81) ==== 1477+0+0 (unknown 0 0 0) 0x46eb500 con 0x3741000 2026-02-11T23:30:34.279+0000 7fac606e4640 20 osd.0 78 OSD::ms_dispatch: osd_map(79..81 src has 1..81) 2026-02-11T23:30:34.279+0000 7fac606e4640 20 osd.0 78 _dispatch 0x46eb500 osd_map(79..81 src has 1..81) 2026-02-11T23:30:34.279+0000 7fac606e4640 3 osd.0 78 handle_osd_map epochs [79,81], i have 78, src has [1,81] 2026-02-11T23:30:34.279+0000 7fac606e4640 10 osd.0 78 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:34.279+0000 7fac606e4640 10 osd.0 78 handle_osd_map got inc map for epoch 79 2026-02-11T23:30:34.279+0000 7fac606e4640 20 osd.0 78 got_full_map 79, nothing requested 2026-02-11T23:30:34.279+0000 7fac606e4640 10 osd.0 78 handle_osd_map got inc map for epoch 80 2026-02-11T23:30:34.279+0000 7fac606e4640 20 osd.0 78 got_full_map 80, nothing requested 2026-02-11T23:30:34.279+0000 7fac606e4640 10 osd.0 78 handle_osd_map got inc map for epoch 81 2026-02-11T23:30:34.279+0000 7fac606e4640 20 osd.0 78 got_full_map 81, nothing requested 2026-02-11T23:30:34.279+0000 7fac606e4640 10 osd.0 78 _track_pools_and_pg_num_changes recording new pool 9 pg_num 1 2026-02-11T23:30:34.279+0000 7fac606e4640 20 osd.0 78 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:34.279+0000 7fac606e4640 20 osd.0 78 handle_osd_map pg_num_history pg_num_history(e81 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1}} deleted_pools ) 2026-02-11T23:30:34.279+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {79={},80={},81={}} 2026-02-11T23:30:34.279+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:34.279+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e81 maps [1~81] lci=[5,81] tlb=1) 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0 78 _committed_osd_maps 79..81 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0 78 advance to epoch 79 (<= last 81 <= newest_map 81) 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0 79 advance to epoch 80 (<= last 81 <= newest_map 81) 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0 80 advance to epoch 81 (<= last 81 <= newest_map 81) 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0 81 consume_map version 81 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0 78 await_reserved_maps epoch:81 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0 78 await_reserved_maps done 0 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0 81 consume_map 81 -- publish done 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0 81 identify_splits_and_merges 4.0 e78 to e81 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0 81 identify_splits_and_merges 4.0 e78 to e81 pg_nums {45=1} 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0 81 identify_splits_and_merges 1.0 e78 to e81 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0 81 identify_splits_and_merges 1.0 e78 to e81 pg_nums {24=1} 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0 81 prune_pg_created 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0:0.consume_map 81 (was 78) 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0:1.consume_map 81 (was 78) 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0:2.consume_map 81 (was 78) 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0:3.consume_map 81 (was 78) 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0:4.consume_map 81 (was 78) 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0:5.consume_map 81 (was 78) 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0:6.consume_map 81 (was 78) 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0:7.consume_map 81 (was 78) 2026-02-11T23:30:34.279+0000 7fac5e983640 10 osd.0 81 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:34.279+0000 7fac5e983640 15 osd.0 81 enqueue_peering_evt 4.0 epoch_sent: 81 epoch_requested: 81 NullEvt 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 81 epoch_requested: 81 NullEvt) class_id 2 prio 255 cost 10 e81) 2026-02-11T23:30:34.279+0000 7fac5e983640 15 osd.0 81 enqueue_peering_evt 1.0 epoch_sent: 81 epoch_requested: 81 NullEvt 2026-02-11T23:30:34.279+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 81 epoch_requested: 81 NullEvt) class_id 2 prio 255 cost 10 e81) 2026-02-11T23:30:34.279+0000 7fac5e983640 7 osd.0 81 activate_map version 81 2026-02-11T23:30:34.279+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {82=} 2026-02-11T23:30:34.279+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 81 epoch_requested: 81 NullEvt) class_id 2immediate prio 255 cost 10 e81) queued 2026-02-11T23:30:34.279+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {82=} 2026-02-11T23:30:34.279+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 81 epoch_requested: 81 NullEvt) class_id 2immediate prio 255 cost 10 e81) pg 0x4856000 2026-02-11T23:30:34.279+0000 7fac55971640 20 osd.0 81 agent_entry empty queue 2026-02-11T23:30:34.279+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:34.279+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 81 epoch_requested: 81 NullEvt) class_id 2immediate prio 255 cost 10 e81) queued 2026-02-11T23:30:34.279+0000 7fac51168640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 79 2026-02-11T23:30:34.279+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:34.279+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 81 epoch_requested: 81 NullEvt) class_id 2immediate prio 255 cost 10 e81) pg 0x40f8000 2026-02-11T23:30:34.279+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 78 -> 79 2026-02-11T23:30:34.279+0000 7fac51168640 10 osd.0 pg_epoch: 78 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:34.279+0000 7fac55170640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 79 2026-02-11T23:30:34.280+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 78 -> 79 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 78 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.280+0000 7fac55170640 20 osd.0 81 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.280+0000 7fac51168640 20 osd.0 81 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 80 2026-02-11T23:30:34.280+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 79 -> 80 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 79 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 80 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 80 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.280+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 79 -> 80 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 80 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 79 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:34.280+0000 7fac55170640 20 osd.0 81 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 80 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 80 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.280+0000 7fac51168640 20 osd.0 81 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 80 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 81 2026-02-11T23:30:34.280+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 80 -> 81 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 80 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 81 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 80 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:34.280+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 80 -> 81 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 80 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.280+0000 7fac55170640 20 osd.0 81 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.280+0000 7fac51168640 20 osd.0 81 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 79..81 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 79..81 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:34.280+0000 7fac55170640 20 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 81 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:34.280+0000 7fac51168640 20 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 81 2026-02-11T23:30:34.280+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:34.280+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:34.280+0000 7fac51168640 10 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 81 epoch_requested: 81 NullEvt 2026-02-11T23:30:34.280+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:34.280+0000 7fac55170640 10 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 81 epoch_requested: 81 NullEvt 2026-02-11T23:30:34.280+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.280+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:34.280+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.329+0000 7fac683ea640 10 osd.0 81 tick_without_osd_lock 2026-02-11T23:30:34.329+0000 7fac683ea640 0 osd.0 81 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:34.329+0000 7fac683ea640 20 osd.0 81 tick_without_osd_lock max_waiting_epoch 82, requesting new map 2026-02-11T23:30:34.329+0000 7fac683ea640 10 monclient: _renew_subs 2026-02-11T23:30:34.329+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:34.329+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=82}) -- 0x44a64e0 con 0x396bc00 2026-02-11T23:30:34.329+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:34.329+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:34.329+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:34.329+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.140 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:34.329+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:34.330, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:34.329+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:34.329+0000 7fac683ea640 10 osd.0 81 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:34.329+0000 7fac683ea640 20 osd.0 81 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:34.329+0000 7fac683ea640 10 osd.0 81 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:34.329+0000 7fac683ea640 20 osd.0 81 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:34.329+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 46 ==== osd_map(82..82 src has 1..82) ==== 296+0+0 (crc 0 0 0) 0x46eb6c0 con 0x396bc00 2026-02-11T23:30:34.329+0000 7fac626e8640 20 osd.0 81 OSD::ms_dispatch: osd_map(82..82 src has 1..82) 2026-02-11T23:30:34.329+0000 7fac626e8640 20 osd.0 81 _dispatch 0x46eb6c0 osd_map(82..82 src has 1..82) 2026-02-11T23:30:34.329+0000 7fac626e8640 3 osd.0 81 handle_osd_map epochs [82,82], i have 81, src has [1,82] 2026-02-11T23:30:34.329+0000 7fac626e8640 10 osd.0 81 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:34.329+0000 7fac626e8640 10 osd.0 81 handle_osd_map got inc map for epoch 82 2026-02-11T23:30:34.329+0000 7fac626e8640 20 osd.0 81 got_full_map 82, nothing requested 2026-02-11T23:30:34.329+0000 7fac626e8640 20 osd.0 81 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:34.329+0000 7fac626e8640 20 osd.0 81 handle_osd_map pg_num_history pg_num_history(e82 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1}} deleted_pools ) 2026-02-11T23:30:34.329+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {82={}} 2026-02-11T23:30:34.329+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:34.329+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e82 maps [1~82] lci=[5,82] tlb=1) 2026-02-11T23:30:34.329+0000 7fac5e983640 10 osd.0 81 _committed_osd_maps 82..82 2026-02-11T23:30:34.329+0000 7fac5e983640 10 osd.0 81 advance to epoch 82 (<= last 82 <= newest_map 82) 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0 82 consume_map version 82 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0 81 await_reserved_maps epoch:82 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0 81 await_reserved_maps done 0 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0 82 consume_map 82 -- publish done 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0 82 identify_splits_and_merges 4.0 e81 to e82 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0 82 identify_splits_and_merges 4.0 e81 to e82 pg_nums {45=1} 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0 82 identify_splits_and_merges 1.0 e81 to e82 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0 82 identify_splits_and_merges 1.0 e81 to e82 pg_nums {24=1} 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0 82 prune_pg_created 2026-02-11T23:30:34.329+0000 7fac5e983640 10 osd.0:0.consume_map 82 (was 81) 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 82 <= 82, requeueing 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {82=} 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:34.329+0000 7fac5e983640 10 osd.0:1.consume_map 82 (was 81) 2026-02-11T23:30:34.329+0000 7fac5e983640 10 osd.0:2.consume_map 82 (was 81) 2026-02-11T23:30:34.329+0000 7fac5e983640 10 osd.0:3.consume_map 82 (was 81) 2026-02-11T23:30:34.329+0000 7fac5e983640 10 osd.0:4.consume_map 82 (was 81) 2026-02-11T23:30:34.329+0000 7fac5e983640 10 osd.0:5.consume_map 82 (was 81) 2026-02-11T23:30:34.329+0000 7fac5e983640 10 osd.0:6.consume_map 82 (was 81) 2026-02-11T23:30:34.329+0000 7fac5e983640 10 osd.0:7.consume_map 82 (was 81) 2026-02-11T23:30:34.329+0000 7fac5e983640 10 osd.0 82 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:34.329+0000 7fac5e983640 15 osd.0 82 enqueue_peering_evt 4.0 epoch_sent: 82 epoch_requested: 82 NullEvt 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 MLease epoch 82 from osd.4 pg_lease(ru 25.104763031s ub 33.104816437s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e82) queued 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 NullEvt) class_id 2 prio 255 cost 10 e82) 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 MLease epoch 82 from osd.4 pg_lease(ru 25.104763031s ub 33.104816437s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e82) pg 0x4856000 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 82 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 81 -> 82 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 81 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:34.329+0000 7fac5e983640 15 osd.0 82 enqueue_peering_evt 1.0 epoch_sent: 82 epoch_requested: 82 NullEvt 2026-02-11T23:30:34.329+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 NullEvt) class_id 2 prio 255 cost 10 e82) 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.329+0000 7fac5e983640 7 osd.0 82 activate_map version 82 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 82 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 82..82 2026-02-11T23:30:34.329+0000 7fac55971640 20 osd.0 82 agent_entry empty queue 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 82 2026-02-11T23:30:34.329+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:34.329+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 82 epoch_requested: 82 MLease epoch 82 from osd.4 pg_lease(ru 25.104763031s ub 33.104816437s int 16.000000000s) 2026-02-11T23:30:34.329+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 NullEvt) class_id 2immediate prio 255 cost 10 e82) queued 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 25.104763031s ub 33.104816437s int 16.000000000s) 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.886744499s -> ru 34.991504669s 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 34.991504669s 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.886969566s -> ruub 42.991786957s 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 42.991786957s 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 82 get_con_osd_cluster to osd.4 from_epoch 82 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 82 get_nextmap_reserved map_reservations: {82=1} 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 82 release_map epoch: 82 2026-02-11T23:30:34.329+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_lease_ack(4.0 pg_lease_ack(ruub 33.104816437s) e82/82) -- 0x25d94a0 con 0x3e4a000 2026-02-11T23:30:34.329+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 NullEvt) class_id 2immediate prio 255 cost 10 e82) queued 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 NullEvt) class_id 2immediate prio 255 cost 10 e82) pg 0x40f8000 2026-02-11T23:30:34.329+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 82 2026-02-11T23:30:34.329+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 82 epoch_requested: 82 NullEvt) class_id 2immediate prio 255 cost 10 e82) pg 0x4856000 2026-02-11T23:30:34.329+0000 7fac55170640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 82 epoch_requested: 82 NullEvt 2026-02-11T23:30:34.329+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 81 -> 82 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 81 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:34.329+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 82 advance_pg new pool opts old pool opts 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 82..82 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 82 2026-02-11T23:30:34.329+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:34.329+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 82 epoch_requested: 82 NullEvt 2026-02-11T23:30:34.329+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:34.329+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:34.955+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 7 ==== osd_ping(ping e82 up_from 21 ping_stamp 2026-02-11T23:30:34.956457+0000/17.903964996s send_stamp 17.903964996s delta_ub -9.886744499s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bc80 con 0x3f57400 2026-02-11T23:30:34.955+0000 7fac703fa640 20 osd.0 82 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886981010s,-9.886744499s]) 2026-02-11T23:30:34.955+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e82 up_from 5 ping_stamp 2026-02-11T23:30:34.956457+0000/17.903964996s send_stamp 27.790946960s delta_ub 9.886981010s) -- 0x3b1a780 con 0x3f57400 2026-02-11T23:30:34.955+0000 7fac703fa640 20 osd.0 82 get_con_osd_cluster to osd.4 from_epoch 82 2026-02-11T23:30:34.955+0000 7fac703fa640 20 osd.0 82 get_nextmap_reserved map_reservations: {82=1} 2026-02-11T23:30:34.955+0000 7fac703fa640 20 osd.0 82 release_map epoch: 82 2026-02-11T23:30:34.955+0000 7fac703fa640 10 osd.0 82 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 68 to ping map epoch of 82 2026-02-11T23:30:34.955+0000 7fac703fa640 10 osd.0 82 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 82 is not newer than session's projected_epoch of 82 2026-02-11T23:30:34.955+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 7 ==== osd_ping(ping e82 up_from 21 ping_stamp 2026-02-11T23:30:34.956457+0000/17.903964996s send_stamp 17.903964996s delta_ub -9.886744499s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a000 con 0x3f57800 2026-02-11T23:30:34.955+0000 7fac71bfd640 20 osd.0 82 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887009621s,-9.886744499s]) 2026-02-11T23:30:34.955+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e82 up_from 5 ping_stamp 2026-02-11T23:30:34.956457+0000/17.903964996s send_stamp 27.790975571s delta_ub 9.887009621s) -- 0x3b1ad80 con 0x3f57800 2026-02-11T23:30:34.955+0000 7fac71bfd640 20 osd.0 82 get_con_osd_cluster to osd.4 from_epoch 82 2026-02-11T23:30:34.955+0000 7fac71bfd640 20 osd.0 82 get_nextmap_reserved map_reservations: {82=1} 2026-02-11T23:30:34.955+0000 7fac71bfd640 20 osd.0 82 release_map epoch: 82 2026-02-11T23:30:34.955+0000 7fac71bfd640 10 osd.0 82 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 82 is not newer than session's projected_epoch of 82 2026-02-11T23:30:35.033+0000 7fac68beb640 10 osd.0 82 tick 2026-02-11T23:30:35.033+0000 7fac68beb640 20 osd.0 82 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:35.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:35.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:35.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:05.168894+0000) 2026-02-11T23:30:35.223+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 47 ==== osd_map(83..84 src has 1..84) ==== 1200+0+0 (crc 0 0 0) 0x46eb880 con 0x396bc00 2026-02-11T23:30:35.223+0000 7fac626e8640 20 osd.0 82 OSD::ms_dispatch: osd_map(83..84 src has 1..84) 2026-02-11T23:30:35.223+0000 7fac626e8640 20 osd.0 82 _dispatch 0x46eb880 osd_map(83..84 src has 1..84) 2026-02-11T23:30:35.223+0000 7fac626e8640 3 osd.0 82 handle_osd_map epochs [83,84], i have 82, src has [1,84] 2026-02-11T23:30:35.223+0000 7fac626e8640 10 osd.0 82 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:35.223+0000 7fac626e8640 10 osd.0 82 handle_osd_map got inc map for epoch 83 2026-02-11T23:30:35.223+0000 7fac626e8640 20 osd.0 82 got_full_map 83, nothing requested 2026-02-11T23:30:35.223+0000 7fac626e8640 10 osd.0 82 handle_osd_map got inc map for epoch 84 2026-02-11T23:30:35.223+0000 7fac626e8640 20 osd.0 82 got_full_map 84, nothing requested 2026-02-11T23:30:35.223+0000 7fac626e8640 20 osd.0 82 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:35.223+0000 7fac626e8640 20 osd.0 82 handle_osd_map pg_num_history pg_num_history(e84 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1}} deleted_pools ) 2026-02-11T23:30:35.223+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {83={},84={}} 2026-02-11T23:30:35.223+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:35.223+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e84 maps [1~84] lci=[5,84] tlb=1) 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0 82 _committed_osd_maps 83..84 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0 82 advance to epoch 83 (<= last 84 <= newest_map 84) 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0 83 advance to epoch 84 (<= last 84 <= newest_map 84) 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0 84 consume_map version 84 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0 82 await_reserved_maps epoch:84 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0 82 await_reserved_maps done 0 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0 84 consume_map 84 -- publish done 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0 84 identify_splits_and_merges 4.0 e82 to e84 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0 84 identify_splits_and_merges 4.0 e82 to e84 pg_nums {45=1} 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0 84 identify_splits_and_merges 1.0 e82 to e84 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0 84 identify_splits_and_merges 1.0 e82 to e84 pg_nums {24=1} 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0 84 prune_pg_created 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0:0.consume_map 84 (was 82) 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0:1.consume_map 84 (was 82) 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0:2.consume_map 84 (was 82) 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0:3.consume_map 84 (was 82) 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0:4.consume_map 84 (was 82) 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0:5.consume_map 84 (was 82) 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0:6.consume_map 84 (was 82) 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0:7.consume_map 84 (was 82) 2026-02-11T23:30:35.224+0000 7fac5e983640 10 osd.0 84 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:35.224+0000 7fac5e983640 15 osd.0 84 enqueue_peering_evt 4.0 epoch_sent: 84 epoch_requested: 84 NullEvt 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2 prio 255 cost 10 e84) 2026-02-11T23:30:35.224+0000 7fac5e983640 15 osd.0 84 enqueue_peering_evt 1.0 epoch_sent: 84 epoch_requested: 84 NullEvt 2026-02-11T23:30:35.224+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2 prio 255 cost 10 e84) 2026-02-11T23:30:35.224+0000 7fac5e983640 7 osd.0 84 activate_map version 84 2026-02-11T23:30:35.224+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.224+0000 7fac55971640 20 osd.0 84 agent_entry empty queue 2026-02-11T23:30:35.224+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2immediate prio 255 cost 10 e84) queued 2026-02-11T23:30:35.224+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.224+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2immediate prio 255 cost 10 e84) pg 0x4856000 2026-02-11T23:30:35.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.224+0000 7fac55170640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 83 2026-02-11T23:30:35.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2immediate prio 255 cost 10 e84) queued 2026-02-11T23:30:35.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 84 epoch_requested: 84 NullEvt) class_id 2immediate prio 255 cost 10 e84) pg 0x40f8000 2026-02-11T23:30:35.224+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 82 -> 83 2026-02-11T23:30:35.224+0000 7fac55170640 10 osd.0 pg_epoch: 82 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:35.224+0000 7fac55170640 10 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.224+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 83 2026-02-11T23:30:35.224+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 82 -> 83 2026-02-11T23:30:35.224+0000 7fac55170640 10 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.224+0000 7fac51168640 10 osd.0 pg_epoch: 82 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:35.224+0000 7fac55170640 20 osd.0 84 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.224+0000 7fac55170640 10 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 84 2026-02-11T23:30:35.224+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 83 -> 84 2026-02-11T23:30:35.224+0000 7fac51168640 10 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.224+0000 7fac55170640 10 osd.0 pg_epoch: 83 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:35.224+0000 7fac55170640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.224+0000 7fac51168640 10 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.224+0000 7fac55170640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.224+0000 7fac55170640 20 osd.0 84 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.224+0000 7fac55170640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 83..84 2026-02-11T23:30:35.224+0000 7fac51168640 20 osd.0 84 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.224+0000 7fac55170640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:35.224+0000 7fac51168640 10 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 84 2026-02-11T23:30:35.224+0000 7fac55170640 20 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 84 2026-02-11T23:30:35.224+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:35.224+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 83 -> 84 2026-02-11T23:30:35.224+0000 7fac55170640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 84 epoch_requested: 84 NullEvt 2026-02-11T23:30:35.224+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:35.224+0000 7fac51168640 10 osd.0 pg_epoch: 83 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:35.224+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.224+0000 7fac51168640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.224+0000 7fac51168640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.224+0000 7fac51168640 20 osd.0 84 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.224+0000 7fac51168640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 83..84 2026-02-11T23:30:35.224+0000 7fac51168640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:35.224+0000 7fac51168640 20 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 84 2026-02-11T23:30:35.224+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:35.224+0000 7fac51168640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 84 epoch_requested: 84 NullEvt 2026-02-11T23:30:35.224+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:35.224+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.317+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 5 ==== osd_ping(ping e82 up_from 13 ping_stamp 2026-02-11T23:30:35.318033+0000/23.503429413s send_stamp 23.503429413s delta_ub -4.648905754s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bb00 con 0x3f56000 2026-02-11T23:30:35.317+0000 7fac723fe640 20 osd.0 84 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649087906s,-4.648905754s]) 2026-02-11T23:30:35.317+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e84 up_from 5 ping_stamp 2026-02-11T23:30:35.318033+0000/23.503429413s send_stamp 28.152517319s delta_ub 4.649087906s) -- 0x3b1b500 con 0x3f56000 2026-02-11T23:30:35.317+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 5 ==== osd_ping(ping e82 up_from 13 ping_stamp 2026-02-11T23:30:35.318033+0000/23.503429413s send_stamp 23.503429413s delta_ub -4.648905754s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a780 con 0x3f56400 2026-02-11T23:30:35.317+0000 7fac723fe640 20 osd.0 84 get_con_osd_cluster to osd.2 from_epoch 84 2026-02-11T23:30:35.317+0000 7fac723fe640 20 osd.0 84 get_nextmap_reserved map_reservations: {84=1} 2026-02-11T23:30:35.317+0000 7fac723fe640 20 osd.0 84 release_map epoch: 84 2026-02-11T23:30:35.317+0000 7fac723fe640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 68 to ping map epoch of 82 2026-02-11T23:30:35.317+0000 7fac723fe640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6820/2662569638 map epoch 82 -> 84 (shared) 2026-02-11T23:30:35.317+0000 7fac723fe640 10 osd.0 84 send_incremental_map epoch range: (83, 84] to v2:127.0.0.1:6820/2662569638 2026-02-11T23:30:35.317+0000 7fac723fe640 10 osd.0 84 add_map_inc_bl 83 574 bytes 2026-02-11T23:30:35.317+0000 7fac723fe640 10 osd.0 84 add_map_inc_bl 84 574 bytes 2026-02-11T23:30:35.317+0000 7fac723fe640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] -- osd_map(83..84 src has 1..84) -- 0x3941500 con 0x3f57c00 2026-02-11T23:30:35.317+0000 7fac703fa640 20 osd.0 84 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649364471s,-4.648905754s]) 2026-02-11T23:30:35.317+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e84 up_from 5 ping_stamp 2026-02-11T23:30:35.318033+0000/23.503429413s send_stamp 28.152793884s delta_ub 4.649364471s) -- 0x3b1bc80 con 0x3f56400 2026-02-11T23:30:35.317+0000 7fac703fa640 20 osd.0 84 get_con_osd_cluster to osd.2 from_epoch 84 2026-02-11T23:30:35.317+0000 7fac703fa640 20 osd.0 84 get_nextmap_reserved map_reservations: {84=1} 2026-02-11T23:30:35.317+0000 7fac703fa640 20 osd.0 84 release_map epoch: 84 2026-02-11T23:30:35.317+0000 7fac703fa640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 84 is not newer than session's projected_epoch of 84 2026-02-11T23:30:35.317+0000 7fac683ea640 10 osd.0 84 tick_without_osd_lock 2026-02-11T23:30:35.317+0000 7fac683ea640 0 osd.0 84 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:35.317+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:35.317+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.129 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:35.317+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:35.318, recovery is active?:false restrictions:<....> 2026-02-11T23:30:35.317+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:35.317+0000 7fac683ea640 10 osd.0 84 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:35.317+0000 7fac683ea640 20 osd.0 84 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:35.317+0000 7fac683ea640 10 osd.0 84 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:35.317+0000 7fac683ea640 20 osd.0 84 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:35.348+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 8 ==== osd_ping(ping e85 up_from 9 ping_stamp 2026-02-11T23:30:35.350198+0000/26.203762054s send_stamp 26.203762054s delta_ub -1.980766416s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ad80 con 0x396a400 2026-02-11T23:30:35.348+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 8 ==== osd_ping(ping e85 up_from 9 ping_stamp 2026-02-11T23:30:35.350198+0000/26.203762054s send_stamp 26.203762054s delta_ub -1.980766416s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b500 con 0x396a000 2026-02-11T23:30:35.348+0000 7fac71bfd640 20 osd.0 84 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980925918s,-1.980766416s]) 2026-02-11T23:30:35.348+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e84 up_from 5 ping_stamp 2026-02-11T23:30:35.350198+0000/26.203762054s send_stamp 28.184688568s delta_ub 1.980925918s) -- 0x3b1a000 con 0x396a400 2026-02-11T23:30:35.348+0000 7fac71bfd640 20 osd.0 84 get_con_osd_cluster to osd.1 from_epoch 84 2026-02-11T23:30:35.348+0000 7fac71bfd640 20 osd.0 84 get_nextmap_reserved map_reservations: {84=1} 2026-02-11T23:30:35.348+0000 7fac71bfd640 20 osd.0 84 release_map epoch: 84 2026-02-11T23:30:35.348+0000 7fac71bfd640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 68 to ping map epoch of 85 2026-02-11T23:30:35.348+0000 7fac71bfd640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 84 is not newer than session's projected_epoch of 85 2026-02-11T23:30:35.348+0000 7fac723fe640 20 osd.0 84 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980946898s,-1.980766416s]) 2026-02-11T23:30:35.348+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e84 up_from 5 ping_stamp 2026-02-11T23:30:35.350198+0000/26.203762054s send_stamp 28.184709549s delta_ub 1.980946898s) -- 0x3b1bb00 con 0x396a000 2026-02-11T23:30:35.348+0000 7fac723fe640 20 osd.0 84 get_con_osd_cluster to osd.1 from_epoch 84 2026-02-11T23:30:35.348+0000 7fac723fe640 20 osd.0 84 get_nextmap_reserved map_reservations: {84=1} 2026-02-11T23:30:35.348+0000 7fac723fe640 20 osd.0 84 release_map epoch: 84 2026-02-11T23:30:35.348+0000 7fac723fe640 10 osd.0 84 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 84 is not newer than session's projected_epoch of 85 2026-02-11T23:30:35.349+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 20 ==== osd_map(85..85 src has 1..85) ==== 626+0+0 (unknown 0 0 0) 0x46eae00 con 0x3752000 2026-02-11T23:30:35.349+0000 7fac606e4640 20 osd.0 84 OSD::ms_dispatch: osd_map(85..85 src has 1..85) 2026-02-11T23:30:35.349+0000 7fac606e4640 20 osd.0 84 _dispatch 0x46eae00 osd_map(85..85 src has 1..85) 2026-02-11T23:30:35.349+0000 7fac606e4640 3 osd.0 84 handle_osd_map epochs [85,85], i have 84, src has [1,85] 2026-02-11T23:30:35.349+0000 7fac606e4640 10 osd.0 84 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:35.349+0000 7fac606e4640 10 osd.0 84 handle_osd_map got inc map for epoch 85 2026-02-11T23:30:35.349+0000 7fac606e4640 20 osd.0 84 got_full_map 85, nothing requested 2026-02-11T23:30:35.349+0000 7fac606e4640 20 osd.0 84 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:35.349+0000 7fac606e4640 20 osd.0 84 handle_osd_map pg_num_history pg_num_history(e85 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1}} deleted_pools ) 2026-02-11T23:30:35.349+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {85={}} 2026-02-11T23:30:35.349+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:35.349+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e85 maps [1~85] lci=[5,85] tlb=1) 2026-02-11T23:30:35.349+0000 7fac5e983640 10 osd.0 84 _committed_osd_maps 85..85 2026-02-11T23:30:35.349+0000 7fac5e983640 10 osd.0 84 advance to epoch 85 (<= last 85 <= newest_map 85) 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0 85 consume_map version 85 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0 84 await_reserved_maps epoch:85 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0 84 await_reserved_maps done 0 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0 85 consume_map 85 -- publish done 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0 85 identify_splits_and_merges 4.0 e84 to e85 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0 85 identify_splits_and_merges 4.0 e84 to e85 pg_nums {45=1} 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0 85 identify_splits_and_merges 1.0 e84 to e85 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0 85 identify_splits_and_merges 1.0 e84 to e85 pg_nums {24=1} 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0 85 prune_pg_created 2026-02-11T23:30:35.349+0000 7fac5e983640 10 osd.0:0.consume_map 85 (was 84) 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:35.349+0000 7fac5e983640 10 osd.0:1.consume_map 85 (was 84) 2026-02-11T23:30:35.349+0000 7fac5e983640 10 osd.0:2.consume_map 85 (was 84) 2026-02-11T23:30:35.349+0000 7fac5e983640 10 osd.0:3.consume_map 85 (was 84) 2026-02-11T23:30:35.349+0000 7fac5e983640 10 osd.0:4.consume_map 85 (was 84) 2026-02-11T23:30:35.349+0000 7fac5e983640 10 osd.0:5.consume_map 85 (was 84) 2026-02-11T23:30:35.349+0000 7fac5e983640 10 osd.0:6.consume_map 85 (was 84) 2026-02-11T23:30:35.349+0000 7fac5e983640 10 osd.0:7.consume_map 85 (was 84) 2026-02-11T23:30:35.349+0000 7fac5e983640 10 osd.0 85 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:35.349+0000 7fac5e983640 15 osd.0 85 enqueue_peering_evt 4.0 epoch_sent: 85 epoch_requested: 85 NullEvt 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2 prio 255 cost 10 e85) 2026-02-11T23:30:35.349+0000 7fac5e983640 15 osd.0 85 enqueue_peering_evt 1.0 epoch_sent: 85 epoch_requested: 85 NullEvt 2026-02-11T23:30:35.349+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2 prio 255 cost 10 e85) 2026-02-11T23:30:35.349+0000 7fac5e983640 7 osd.0 85 activate_map version 85 2026-02-11T23:30:35.349+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.349+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2immediate prio 255 cost 10 e85) queued 2026-02-11T23:30:35.349+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.349+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2immediate prio 255 cost 10 e85) pg 0x4856000 2026-02-11T23:30:35.349+0000 7fac55971640 20 osd.0 85 agent_entry empty queue 2026-02-11T23:30:35.349+0000 7fac55170640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 85 2026-02-11T23:30:35.349+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:35.349+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2immediate prio 255 cost 10 e85) queued 2026-02-11T23:30:35.349+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:35.349+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 85 epoch_requested: 85 NullEvt) class_id 2immediate prio 255 cost 10 e85) pg 0x40f8000 2026-02-11T23:30:35.349+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 84 -> 85 2026-02-11T23:30:35.349+0000 7fac55170640 10 osd.0 pg_epoch: 84 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:35.349+0000 7fac55170640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.349+0000 7fac55170640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.349+0000 7fac55170640 20 osd.0 85 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.349+0000 7fac51168640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 85 2026-02-11T23:30:35.349+0000 7fac55170640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 85..85 2026-02-11T23:30:35.349+0000 7fac55170640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:35.349+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 84 -> 85 2026-02-11T23:30:35.349+0000 7fac55170640 20 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 85 2026-02-11T23:30:35.349+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:35.349+0000 7fac55170640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 85 epoch_requested: 85 NullEvt 2026-02-11T23:30:35.349+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:35.349+0000 7fac51168640 10 osd.0 pg_epoch: 84 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:35.349+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:35.349+0000 7fac51168640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:35.349+0000 7fac51168640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:35.349+0000 7fac51168640 20 osd.0 85 advance_pg new pool opts old pool opts 2026-02-11T23:30:35.349+0000 7fac51168640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 85..85 2026-02-11T23:30:35.349+0000 7fac51168640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:35.349+0000 7fac51168640 20 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 85 2026-02-11T23:30:35.349+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:35.349+0000 7fac51168640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 85 epoch_requested: 85 NullEvt 2026-02-11T23:30:35.349+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:35.349+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.025+0000 7fac68beb640 10 osd.0 85 tick 2026-02-11T23:30:36.025+0000 7fac68beb640 20 osd.0 85 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:36.055+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 8 ==== osd_ping(ping e85 up_from 21 ping_stamp 2026-02-11T23:30:36.056764+0000/19.004270554s send_stamp 19.004270554s delta_ub -9.886837959s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a000 con 0x3f57800 2026-02-11T23:30:36.055+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 8 ==== osd_ping(ping e85 up_from 21 ping_stamp 2026-02-11T23:30:36.056764+0000/19.004270554s send_stamp 19.004270554s delta_ub -9.886837959s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bc80 con 0x3f57400 2026-02-11T23:30:36.055+0000 7fac71bfd640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887051582s,-9.886837959s]) 2026-02-11T23:30:36.055+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e85 up_from 5 ping_stamp 2026-02-11T23:30:36.056764+0000/19.004270554s send_stamp 28.891323090s delta_ub 9.887051582s) -- 0x3b1ad80 con 0x3f57800 2026-02-11T23:30:36.055+0000 7fac71bfd640 20 osd.0 85 get_con_osd_cluster to osd.4 from_epoch 85 2026-02-11T23:30:36.055+0000 7fac71bfd640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:36.055+0000 7fac71bfd640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:36.055+0000 7fac71bfd640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 82 to ping map epoch of 85 2026-02-11T23:30:36.055+0000 7fac71bfd640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 85 is not newer than session's projected_epoch of 85 2026-02-11T23:30:36.055+0000 7fac703fa640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887080193s,-9.886837959s]) 2026-02-11T23:30:36.055+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e85 up_from 5 ping_stamp 2026-02-11T23:30:36.056764+0000/19.004270554s send_stamp 28.891351700s delta_ub 9.887080193s) -- 0x3b1a780 con 0x3f57400 2026-02-11T23:30:36.055+0000 7fac703fa640 20 osd.0 85 get_con_osd_cluster to osd.4 from_epoch 85 2026-02-11T23:30:36.055+0000 7fac703fa640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:36.055+0000 7fac703fa640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:36.055+0000 7fac703fa640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 85 is not newer than session's projected_epoch of 85 2026-02-11T23:30:36.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:36.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:36.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:06.168929+0000) 2026-02-11T23:30:36.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 21 ==== pg_lease(1.0 pg_lease(ru 35.071456909s ub 43.071571350s int 16.000000000s) e87/87) ==== 58+0+0 (unknown 0 0 0) 0x3b1a780 con 0x3752000 2026-02-11T23:30:36.217+0000 7fac703fa640 15 osd.0 85 enqueue_peering_evt 1.0 epoch_sent: 87 epoch_requested: 87 MLease epoch 87 from osd.1 pg_lease(ru 35.071456909s ub 43.071571350s int 16.000000000s) 2026-02-11T23:30:36.217+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 MLease epoch 87 from osd.1 pg_lease(ru 35.071456909s ub 43.071571350s int 16.000000000s)) class_id 2 prio 255 cost 10 e87) 2026-02-11T23:30:36.217+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.217+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 MLease epoch 87 from osd.1 pg_lease(ru 35.071456909s ub 43.071571350s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e87) queued 2026-02-11T23:30:36.217+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.217+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 MLease epoch 87 from osd.1 pg_lease(ru 35.071456909s ub 43.071571350s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e87) pg 0x40f8000 2026-02-11T23:30:36.217+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 87, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 MLease epoch 87 from osd.1 pg_lease(ru 35.071456909s ub 43.071571350s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e87) 2026-02-11T23:30:36.217+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.269+0000 7fac4d160640 5 osd.0 85 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x5bf66/0xd2000, compress 0x0/0x0/0x0, omap 0x82f4, meta 0x1a47d0c), peers [1,2,3,4] op hist []) 2026-02-11T23:30:36.269+0000 7fac4d160640 20 osd.0 85 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:36.269+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e85 up_from 5 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 29.104541779s delta_ub 1.980946898s) -- 0x3d0a000 con 0x396b400 2026-02-11T23:30:36.269+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e85 up_from 5 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 29.104541779s delta_ub 1.980946898s) -- 0x3d0a180 con 0x3752400 2026-02-11T23:30:36.269+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e85 up_from 5 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 29.104541779s delta_ub 4.649364471s) -- 0x3d0a300 con 0x396b800 2026-02-11T23:30:36.269+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e85 up_from 5 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 29.104541779s delta_ub 4.649364471s) -- 0x3d0a480 con 0x34c1000 2026-02-11T23:30:36.269+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e85 up_from 5 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 29.104541779s delta_ub 7.310496330s) -- 0x3d0a600 con 0x3740000 2026-02-11T23:30:36.269+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e85 up_from 5 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 29.104541779s delta_ub 7.310496330s) -- 0x3d0a780 con 0x3740400 2026-02-11T23:30:36.269+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e85 up_from 5 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 29.104541779s delta_ub 9.887080193s) -- 0x3d0a900 con 0x3740800 2026-02-11T23:30:36.269+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e85 up_from 5 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 29.104541779s delta_ub 9.887080193s) -- 0x3d0aa80 con 0x3740c00 2026-02-11T23:30:36.269+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 8 ==== osd_ping(ping_reply e87 up_from 9 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 27.123764038s delta_ub -1.980776787s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a000 con 0x396b400 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 get_con_osd_cluster to osd.1 from_epoch 85 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:36.269+0000 7fac703fa640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 85 to ping map epoch of 87 2026-02-11T23:30:36.269+0000 7fac703fa640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 85 is not newer than session's projected_epoch of 87 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980916977s,-1.980776787s]) 2026-02-11T23:30:36.269+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 8 ==== osd_ping(ping_reply e87 up_from 9 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 27.123794556s delta_ub -1.980746388s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a480 con 0x3752400 2026-02-11T23:30:36.269+0000 7fac71bfd640 20 osd.0 85 get_con_osd_cluster to osd.1 from_epoch 85 2026-02-11T23:30:36.269+0000 7fac71bfd640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:36.269+0000 7fac71bfd640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:36.269+0000 7fac71bfd640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 85 is not newer than session's projected_epoch of 87 2026-02-11T23:30:36.269+0000 7fac71bfd640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980913877s,-1.980746388s]) 2026-02-11T23:30:36.269+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 5 ==== osd_ping(ping_reply e88 up_from 17 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 21.794292450s delta_ub -7.310248852s) ==== 2033+0+0 (crc 0 0 0) 0x3d0aa80 con 0x3740000 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 get_con_osd_cluster to osd.3 from_epoch 85 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:36.269+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 22 ==== osd_map(86..87 src has 1..87) ==== 1200+0+0 (unknown 0 0 0) 0x38e3880 con 0x3752000 2026-02-11T23:30:36.269+0000 7fac703fa640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 81 to ping map epoch of 88 2026-02-11T23:30:36.269+0000 7fac703fa640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 85 is not newer than session's projected_epoch of 88 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 OSD::ms_dispatch: osd_map(86..87 src has 1..87) 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310457230s,-7.310248852s]) 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 _dispatch 0x38e3880 osd_map(86..87 src has 1..87) 2026-02-11T23:30:36.269+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 7 ==== osd_ping(ping_reply e85 up_from 13 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 24.455652237s delta_ub -4.648889065s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a900 con 0x396b800 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 get_con_osd_cluster to osd.2 from_epoch 85 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:36.269+0000 7fac723fe640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 84 to ping map epoch of 85 2026-02-11T23:30:36.269+0000 7fac723fe640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 85 is not newer than session's projected_epoch of 85 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649108410s,-4.648889065s]) 2026-02-11T23:30:36.269+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 7 ==== osd_ping(ping_reply e85 up_from 13 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 24.455690384s delta_ub -4.648850918s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a480 con 0x34c1000 2026-02-11T23:30:36.269+0000 7fac71bfd640 20 osd.0 85 get_con_osd_cluster to osd.2 from_epoch 85 2026-02-11T23:30:36.269+0000 7fac71bfd640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:36.269+0000 7fac71bfd640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:36.269+0000 7fac71bfd640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 85 is not newer than session's projected_epoch of 85 2026-02-11T23:30:36.269+0000 7fac71bfd640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649088860s,-4.648850918s]) 2026-02-11T23:30:36.269+0000 7fac606e4640 3 osd.0 85 handle_osd_map epochs [86,87], i have 85, src has [1,87] 2026-02-11T23:30:36.269+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 5 ==== osd_ping(ping_reply e88 up_from 17 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 21.794332504s delta_ub -7.310208797s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a900 con 0x3740400 2026-02-11T23:30:36.269+0000 7fac606e4640 10 osd.0 85 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:36.269+0000 7fac606e4640 10 osd.0 85 handle_osd_map got inc map for epoch 86 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 get_con_osd_cluster to osd.3 from_epoch 85 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:36.269+0000 7fac723fe640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 85 is not newer than session's projected_epoch of 88 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310480595s,-7.310208797s]) 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 got_full_map 86, nothing requested 2026-02-11T23:30:36.269+0000 7fac606e4640 10 osd.0 85 handle_osd_map got inc map for epoch 87 2026-02-11T23:30:36.269+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 5 ==== osd_ping(ping_reply e88 up_from 21 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 19.217788696s delta_ub -9.886753082s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a900 con 0x3740800 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 get_con_osd_cluster to osd.4 from_epoch 85 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:36.269+0000 7fac723fe640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 85 to ping map epoch of 88 2026-02-11T23:30:36.269+0000 7fac723fe640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 85 is not newer than session's projected_epoch of 88 2026-02-11T23:30:36.269+0000 7fac723fe640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887171745s,-9.886753082s]) 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 got_full_map 87, nothing requested 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 handle_osd_map pg_num_history pg_num_history(e87 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1}} deleted_pools ) 2026-02-11T23:30:36.269+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 5 ==== osd_ping(ping_reply e88 up_from 21 ping_stamp 2026-02-11T23:30:36.270125+0000/29.104541779s send_stamp 19.218011856s delta_ub -9.886527061s) ==== 2033+0+0 (crc 0 0 0) 0x3d0aa80 con 0x3740c00 2026-02-11T23:30:36.269+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {86={},87={}} 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 get_con_osd_cluster to osd.4 from_epoch 85 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 get_nextmap_reserved map_reservations: {85=1} 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 release_map epoch: 85 2026-02-11T23:30:36.269+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:36.269+0000 7fac703fa640 10 osd.0 85 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 85 is not newer than session's projected_epoch of 88 2026-02-11T23:30:36.269+0000 7fac703fa640 20 osd.0 85 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886990547s,-9.886527061s]) 2026-02-11T23:30:36.269+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e87 maps [1~87] lci=[5,87] tlb=1) 2026-02-11T23:30:36.269+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.3 v2:127.0.0.1:6828/1114810440 23 ==== osd_map(86..88 src has 1..88) ==== 1800+0+0 (unknown 0 0 0) 0x38e2700 con 0x3741000 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 OSD::ms_dispatch: osd_map(86..88 src has 1..88) 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 _dispatch 0x38e2700 osd_map(86..88 src has 1..88) 2026-02-11T23:30:36.269+0000 7fac606e4640 3 osd.0 85 handle_osd_map epochs [86,88], i have 87, src has [1,88] 2026-02-11T23:30:36.269+0000 7fac606e4640 10 osd.0 85 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:36.269+0000 7fac606e4640 10 osd.0 85 handle_osd_map got inc map for epoch 88 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 got_full_map 88, nothing requested 2026-02-11T23:30:36.269+0000 7fac606e4640 10 osd.0 85 _track_pools_and_pg_num_changes recording new pool 10 pg_num 1 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 handle_osd_map pg_num_history pg_num_history(e88 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools ) 2026-02-11T23:30:36.269+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {88={}} 2026-02-11T23:30:36.269+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:36.269+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e88 maps [1~88] lci=[5,88] tlb=1) 2026-02-11T23:30:36.269+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 29 ==== osd_map(86..88 src has 1..88) ==== 1800+0+0 (unknown 0 0 0) 0x39416c0 con 0x3e4a000 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 OSD::ms_dispatch: osd_map(86..88 src has 1..88) 2026-02-11T23:30:36.269+0000 7fac606e4640 20 osd.0 85 _dispatch 0x39416c0 osd_map(86..88 src has 1..88) 2026-02-11T23:30:36.269+0000 7fac606e4640 3 osd.0 85 handle_osd_map epochs [86,88], i have 88, src has [1,88] 2026-02-11T23:30:36.269+0000 7fac606e4640 10 osd.0 85 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:36.269+0000 7fac606e4640 10 osd.0 85 no new maps here, dropping 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0 85 _committed_osd_maps 86..87 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0 85 advance to epoch 86 (<= last 87 <= newest_map 88) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0 86 advance to epoch 87 (<= last 87 <= newest_map 88) 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 87 consume_map version 87 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 85 await_reserved_maps epoch:87 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 85 await_reserved_maps done 0 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 87 consume_map 87 -- publish done 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 87 identify_splits_and_merges 4.0 e85 to e87 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 87 identify_splits_and_merges 4.0 e85 to e87 pg_nums {45=1} 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 87 identify_splits_and_merges 1.0 e85 to e87 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 87 identify_splits_and_merges 1.0 e85 to e87 pg_nums {24=1} 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 87 prune_pg_created 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:0.consume_map 87 (was 85) 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 87 <= 87, requeueing 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {87=} 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:1.consume_map 87 (was 85) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:2.consume_map 87 (was 85) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:3.consume_map 87 (was 85) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:4.consume_map 87 (was 85) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:5.consume_map 87 (was 85) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:6.consume_map 87 (was 85) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:7.consume_map 87 (was 85) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0 87 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.269+0000 7fac5e983640 15 osd.0 87 enqueue_peering_evt 4.0 epoch_sent: 87 epoch_requested: 87 NullEvt 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 MLease epoch 87 from osd.1 pg_lease(ru 35.071456909s ub 43.071571350s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e87) queued 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 MLease epoch 87 from osd.1 pg_lease(ru 35.071456909s ub 43.071571350s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e87) pg 0x40f8000 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 NullEvt) class_id 2 prio 255 cost 10 e87) 2026-02-11T23:30:36.269+0000 7fac5e983640 15 osd.0 87 enqueue_peering_evt 1.0 epoch_sent: 87 epoch_requested: 87 NullEvt 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 NullEvt) class_id 2 prio 255 cost 10 e87) 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 NullEvt) class_id 2immediate prio 255 cost 10 e87) queued 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 NullEvt) class_id 2immediate prio 255 cost 10 e87) pg 0x4856000 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 86 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 85 -> 86 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 85 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:36.269+0000 7fac5e983640 7 osd.0 87 activate_map version 87 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 86 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 85 -> 86 2026-02-11T23:30:36.269+0000 7fac55971640 20 osd.0 87 agent_entry empty queue 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 87 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 85 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 87 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 86 -> 87 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 86 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 87 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 86..87 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 87 2026-02-11T23:30:36.269+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 87 epoch_requested: 87 MLease epoch 87 from osd.1 pg_lease(ru 35.071456909s ub 43.071571350s int 16.000000000s) 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 87 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 35.071456909s ub 43.071571350s int 16.000000000s) 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 87 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -1.980746388s -> ru 37.052204132s 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 86 -> 87 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 37.052204132s 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 86 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.980913877s -> ruub 45.052482605s 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 45.052482605s 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 87 get_con_osd_cluster to osd.1 from_epoch 87 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 87 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 87 get_nextmap_reserved map_reservations: {87=1} 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 86..87 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 87 release_map epoch: 87 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 87 2026-02-11T23:30:36.269+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:36.269+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_lease_ack(1.0 pg_lease_ack(ruub 43.071571350s) e87/87) -- 0x25d9340 con 0x3752000 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 87 epoch_requested: 87 NullEvt 2026-02-11T23:30:36.269+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 NullEvt) class_id 2immediate prio 255 cost 10 e87) queued 2026-02-11T23:30:36.269+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 87 epoch_requested: 87 NullEvt) class_id 2immediate prio 255 cost 10 e87) pg 0x40f8000 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 87 epoch_requested: 87 NullEvt 2026-02-11T23:30:36.269+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0 87 _committed_osd_maps 88..88 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0 87 advance to epoch 88 (<= last 88 <= newest_map 88) 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 88 consume_map version 88 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 87 await_reserved_maps epoch:88 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 87 await_reserved_maps done 0 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 88 consume_map 88 -- publish done 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 88 identify_splits_and_merges 4.0 e87 to e88 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 88 identify_splits_and_merges 4.0 e87 to e88 pg_nums {45=1} 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 88 identify_splits_and_merges 1.0 e87 to e88 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 88 identify_splits_and_merges 1.0 e87 to e88 pg_nums {24=1} 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 88 prune_pg_created 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:0.consume_map 88 (was 87) 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:1.consume_map 88 (was 87) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:2.consume_map 88 (was 87) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:3.consume_map 88 (was 87) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:4.consume_map 88 (was 87) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:5.consume_map 88 (was 87) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:6.consume_map 88 (was 87) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0:7.consume_map 88 (was 87) 2026-02-11T23:30:36.269+0000 7fac5e983640 10 osd.0 88 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:36.269+0000 7fac5e983640 15 osd.0 88 enqueue_peering_evt 4.0 epoch_sent: 88 epoch_requested: 88 NullEvt 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 88 epoch_requested: 88 NullEvt) class_id 2 prio 255 cost 10 e88) 2026-02-11T23:30:36.269+0000 7fac5e983640 15 osd.0 88 enqueue_peering_evt 1.0 epoch_sent: 88 epoch_requested: 88 NullEvt 2026-02-11T23:30:36.269+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 88 epoch_requested: 88 NullEvt) class_id 2 prio 255 cost 10 e88) 2026-02-11T23:30:36.269+0000 7fac5e983640 7 osd.0 88 activate_map version 88 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 88 epoch_requested: 88 NullEvt) class_id 2immediate prio 255 cost 10 e88) queued 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 88 epoch_requested: 88 NullEvt) class_id 2immediate prio 255 cost 10 e88) pg 0x4856000 2026-02-11T23:30:36.269+0000 7fac55971640 20 osd.0 88 agent_entry empty queue 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 88 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 88 epoch_requested: 88 NullEvt) class_id 2immediate prio 255 cost 10 e88) queued 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 88 epoch_requested: 88 NullEvt) class_id 2immediate prio 255 cost 10 e88) pg 0x40f8000 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 87 -> 88 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 87 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 88 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 88 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 88..88 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 88 2026-02-11T23:30:36.269+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:36.269+0000 7fac51168640 10 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 88 epoch_requested: 88 NullEvt 2026-02-11T23:30:36.269+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 87 -> 88 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 87 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:36.269+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 88 advance_pg new pool opts old pool opts 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 88..88 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 88 2026-02-11T23:30:36.269+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:36.269+0000 7fac55170640 10 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 88 epoch_requested: 88 NullEvt 2026-02-11T23:30:36.269+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:36.269+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.323+0000 7fac683ea640 10 osd.0 88 tick_without_osd_lock 2026-02-11T23:30:36.323+0000 7fac683ea640 0 osd.0 88 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:36.323+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:36.323+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:36.323+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:36.323+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.129 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:36.323+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:36.324, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:36.323+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:36.323+0000 7fac683ea640 10 osd.0 88 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:36.323+0000 7fac683ea640 20 osd.0 88 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:36.323+0000 7fac683ea640 10 osd.0 88 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:36.323+0000 7fac683ea640 20 osd.0 88 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:36.449+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 9 ==== osd_ping(ping e87 up_from 9 ping_stamp 2026-02-11T23:30:36.450364+0000/27.303928375s send_stamp 27.303928375s delta_ub -1.980746388s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a480 con 0x396a400 2026-02-11T23:30:36.449+0000 7fac71bfd640 20 osd.0 88 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980921865s,-1.980746388s]) 2026-02-11T23:30:36.449+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e88 up_from 5 ping_stamp 2026-02-11T23:30:36.450364+0000/27.303928375s send_stamp 29.284851074s delta_ub 1.980921865s) -- 0x3d0a180 con 0x396a400 2026-02-11T23:30:36.449+0000 7fac71bfd640 20 osd.0 88 get_con_osd_cluster to osd.1 from_epoch 88 2026-02-11T23:30:36.449+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 9 ==== osd_ping(ping e87 up_from 9 ping_stamp 2026-02-11T23:30:36.450364+0000/27.303928375s send_stamp 27.303928375s delta_ub -1.980746388s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a900 con 0x396a000 2026-02-11T23:30:36.449+0000 7fac71bfd640 20 osd.0 88 get_nextmap_reserved map_reservations: {88=1} 2026-02-11T23:30:36.449+0000 7fac71bfd640 20 osd.0 88 release_map epoch: 88 2026-02-11T23:30:36.449+0000 7fac71bfd640 10 osd.0 88 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 87 -> 88 (shared) 2026-02-11T23:30:36.449+0000 7fac71bfd640 10 osd.0 88 send_incremental_map epoch range: (88, 88] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:30:36.449+0000 7fac71bfd640 10 osd.0 88 add_map_inc_bl 88 592 bytes 2026-02-11T23:30:36.449+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(88..88 src has 1..88) -- 0x3941500 con 0x3752000 2026-02-11T23:30:36.449+0000 7fac723fe640 20 osd.0 88 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.981085181s,-1.980746388s]) 2026-02-11T23:30:36.449+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e88 up_from 5 ping_stamp 2026-02-11T23:30:36.450364+0000/27.303928375s send_stamp 29.285013199s delta_ub 1.981085181s) -- 0x3d0a780 con 0x396a000 2026-02-11T23:30:36.449+0000 7fac723fe640 20 osd.0 88 get_con_osd_cluster to osd.1 from_epoch 88 2026-02-11T23:30:36.449+0000 7fac723fe640 20 osd.0 88 get_nextmap_reserved map_reservations: {88=1} 2026-02-11T23:30:36.449+0000 7fac723fe640 20 osd.0 88 release_map epoch: 88 2026-02-11T23:30:36.449+0000 7fac723fe640 10 osd.0 88 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 88 is not newer than session's projected_epoch of 88 2026-02-11T23:30:36.669+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.2 v2:127.0.0.1:6820/2662569638 5 ==== pg_query2(10.0 10.0 query(info 0'0 epoch_sent 88) e88/88) ==== 158+0+0 (unknown 0 0 0) 0x3f3eb40 con 0x3f57c00 2026-02-11T23:30:36.669+0000 7fac71bfd640 15 osd.0 88 enqueue_peering_evt 10.0 epoch_sent: 88 epoch_requested: 88 MQuery 10.0 from 2 query_epoch 88 query: query(info 0'0 epoch_sent 88) 2026-02-11T23:30:36.669+0000 7fac71bfd640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 88 epoch_requested: 88 MQuery 10.0 from 2 query_epoch 88 query: query(info 0'0 epoch_sent 88)) class_id 2 prio 255 cost 10 e88) 2026-02-11T23:30:36.669+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.669+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 88 epoch_requested: 88 MQuery 10.0 from 2 query_epoch 88 query: query(info 0'0 epoch_sent 88)) class_id 2immediate prio 255 cost 10 e88) queued 2026-02-11T23:30:36.669+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.669+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 88 epoch_requested: 88 MQuery 10.0 from 2 query_epoch 88 query: query(info 0'0 epoch_sent 88)) class_id 2immediate prio 255 cost 10 e88) pg 0 2026-02-11T23:30:36.669+0000 7fac51168640 10 osd.0 88 handle_pg_query_nopg 10.0 2026-02-11T23:30:36.669+0000 7fac51168640 10 osd.0 88 pg 10.0 dne 2026-02-11T23:30:36.669+0000 7fac51168640 20 osd.0 88 get_con_osd_cluster to osd.2 from_epoch 88 2026-02-11T23:30:36.669+0000 7fac51168640 20 osd.0 88 get_nextmap_reserved map_reservations: {88=1} 2026-02-11T23:30:36.669+0000 7fac51168640 20 osd.0 88 release_map epoch: 88 2026-02-11T23:30:36.669+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.2 v2:127.0.0.1:6820/2662569638 6 ==== osd_map(86..88 src has 1..88) ==== 1800+0+0 (unknown 0 0 0) 0x3941dc0 con 0x3f57c00 2026-02-11T23:30:36.669+0000 7fac51168640 10 osd.0 88 maybe_share_map: con v2:127.0.0.1:6820/2662569638 map epoch 85 -> 88 (shared) 2026-02-11T23:30:36.669+0000 7fac606e4640 20 osd.0 88 OSD::ms_dispatch: osd_map(86..88 src has 1..88) 2026-02-11T23:30:36.669+0000 7fac606e4640 20 osd.0 88 _dispatch 0x3941dc0 osd_map(86..88 src has 1..88) 2026-02-11T23:30:36.669+0000 7fac51168640 10 osd.0 88 send_incremental_map epoch range: (86, 88] to v2:127.0.0.1:6820/2662569638 2026-02-11T23:30:36.669+0000 7fac51168640 10 osd.0 88 add_map_inc_bl 86 574 bytes 2026-02-11T23:30:36.669+0000 7fac51168640 10 osd.0 88 add_map_inc_bl 87 574 bytes 2026-02-11T23:30:36.669+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] -- osd_map(86..88 src has 1..88) -- 0x3940fc0 con 0x3f57c00 2026-02-11T23:30:36.669+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] -- pg_notify2(10.0 10.0 (query:88 sent:88 10.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e88/88) -- 0x3381500 con 0x3f57c00 2026-02-11T23:30:36.669+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.669+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.669+0000 7fac606e4640 3 osd.0 88 handle_osd_map epochs [86,88], i have 88, src has [1,88] 2026-02-11T23:30:36.669+0000 7fac606e4640 10 osd.0 88 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:36.669+0000 7fac606e4640 10 osd.0 88 no new maps here, dropping 2026-02-11T23:30:36.721+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.2 v2:127.0.0.1:6820/2662569638 7 ==== PGlog(10.0 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.908458710s int 16.000000000s) e89/88) ==== 1172+0+0 (unknown 0 0 0) 0x3e40000 con 0x3f57c00 2026-02-11T23:30:36.721+0000 7fac71bfd640 15 osd.0 88 enqueue_peering_evt 10.0 epoch_sent: 89 epoch_requested: 88 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.908458710s int 16.000000000s) +create_info 2026-02-11T23:30:36.721+0000 7fac71bfd640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 89 epoch_requested: 88 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.908458710s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e89) 2026-02-11T23:30:36.721+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:36.721+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 89 epoch_requested: 88 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.908458710s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e89) queued 2026-02-11T23:30:36.721+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:36.721+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 89 epoch_requested: 88 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.908458710s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e89) pg 0 2026-02-11T23:30:36.721+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 map 89 > 88 2026-02-11T23:30:36.721+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 10.0 peering, item epoch is 89, will wait on OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 89 epoch_requested: 88 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.908458710s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e89) 2026-02-11T23:30:36.721+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:36.721+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.058+0000 7fac68beb640 10 osd.0 88 tick 2026-02-11T23:30:37.058+0000 7fac68beb640 20 osd.0 88 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:37.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:37.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:37.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:07.168981+0000) 2026-02-11T23:30:37.368+0000 7fac683ea640 10 osd.0 88 tick_without_osd_lock 2026-02-11T23:30:37.368+0000 7fac683ea640 0 osd.0 88 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:37.368+0000 7fac683ea640 20 osd.0 88 tick_without_osd_lock max_waiting_epoch 89, requesting new map 2026-02-11T23:30:37.368+0000 7fac683ea640 10 monclient: _renew_subs 2026-02-11T23:30:37.368+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:37.368+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=89}) -- 0x3e56680 con 0x396bc00 2026-02-11T23:30:37.368+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:37.368+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:37.368+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:37.368+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.129 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:37.368+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:37.369, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:37.368+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:37.368+0000 7fac683ea640 10 osd.0 88 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:37.368+0000 7fac683ea640 20 osd.0 88 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:37.368+0000 7fac683ea640 10 osd.0 88 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:37.368+0000 7fac683ea640 20 osd.0 88 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:37.368+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 48 ==== osd_map(89..91 src has 1..91) ==== 1460+0+0 (crc 0 0 0) 0x46eaa80 con 0x396bc00 2026-02-11T23:30:37.368+0000 7fac626e8640 20 osd.0 88 OSD::ms_dispatch: osd_map(89..91 src has 1..91) 2026-02-11T23:30:37.368+0000 7fac626e8640 20 osd.0 88 _dispatch 0x46eaa80 osd_map(89..91 src has 1..91) 2026-02-11T23:30:37.368+0000 7fac626e8640 3 osd.0 88 handle_osd_map epochs [89,91], i have 88, src has [1,91] 2026-02-11T23:30:37.368+0000 7fac626e8640 10 osd.0 88 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:37.368+0000 7fac626e8640 10 osd.0 88 handle_osd_map got inc map for epoch 89 2026-02-11T23:30:37.368+0000 7fac626e8640 20 osd.0 88 got_full_map 89, nothing requested 2026-02-11T23:30:37.368+0000 7fac626e8640 10 osd.0 88 handle_osd_map got inc map for epoch 90 2026-02-11T23:30:37.368+0000 7fac626e8640 20 osd.0 88 got_full_map 90, nothing requested 2026-02-11T23:30:37.368+0000 7fac626e8640 10 osd.0 88 handle_osd_map got inc map for epoch 91 2026-02-11T23:30:37.368+0000 7fac626e8640 20 osd.0 88 got_full_map 91, nothing requested 2026-02-11T23:30:37.368+0000 7fac626e8640 20 osd.0 88 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:37.368+0000 7fac626e8640 20 osd.0 88 handle_osd_map pg_num_history pg_num_history(e91 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools ) 2026-02-11T23:30:37.368+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {89={},90={},91={}} 2026-02-11T23:30:37.368+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:37.368+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e91 maps [1~91] lci=[5,91] tlb=1) 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0 88 _committed_osd_maps 89..91 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0 88 advance to epoch 89 (<= last 91 <= newest_map 91) 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0 89 advance to epoch 90 (<= last 91 <= newest_map 91) 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0 90 advance to epoch 91 (<= last 91 <= newest_map 91) 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0 91 consume_map version 91 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0 88 await_reserved_maps epoch:91 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0 88 await_reserved_maps done 0 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0 91 consume_map 91 -- publish done 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:10.0slot:0x2581600 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 10.0 has no pg and waiting_for_split 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0 91 identify_splits_and_merges 4.0 e88 to e91 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0 91 identify_splits_and_merges 4.0 e88 to e91 pg_nums {45=1} 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0 91 identify_splits_and_merges 1.0 e88 to e91 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0 91 identify_splits_and_merges 1.0 e88 to e91 pg_nums {24=1} 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0 91 prune_pg_created 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0:0.consume_map 91 (was 88) 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 10.0 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 10.0 pending_peering first epoch 89 <= 91, requeueing 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 10.0 to_process <> waiting <> waiting_peering {89=} 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0:1.consume_map 91 (was 88) 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0:2.consume_map 91 (was 88) 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0:3.consume_map 91 (was 88) 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0:4.consume_map 91 (was 88) 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0:5.consume_map 91 (was 88) 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0:6.consume_map 91 (was 88) 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0:7.consume_map 91 (was 88) 2026-02-11T23:30:37.369+0000 7fac5e983640 10 osd.0 91 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.369+0000 7fac5e983640 15 osd.0 91 enqueue_peering_evt 4.0 epoch_sent: 91 epoch_requested: 91 NullEvt 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 89 epoch_requested: 88 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.908458710s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e89) queued 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 89 epoch_requested: 88 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.908458710s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e89) pg 0 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 91 epoch_requested: 91 NullEvt) class_id 2 prio 255 cost 10 e91) 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 no pg, peering, doesn't map here e91, discarding OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 89 epoch_requested: 88 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 40.908458710s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e89) 2026-02-11T23:30:37.369+0000 7fac5e983640 15 osd.0 91 enqueue_peering_evt 1.0 epoch_sent: 91 epoch_requested: 91 NullEvt 2026-02-11T23:30:37.369+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 91 epoch_requested: 91 NullEvt) class_id 2 prio 255 cost 10 e91) 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 91 epoch_requested: 91 NullEvt) class_id 2immediate prio 255 cost 10 e91) queued 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 91 epoch_requested: 91 NullEvt) class_id 2immediate prio 255 cost 10 e91) pg 0x4856000 2026-02-11T23:30:37.369+0000 7fac5e983640 7 osd.0 91 activate_map version 91 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.369+0000 7fac55971640 20 osd.0 91 agent_entry empty queue 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 91 epoch_requested: 91 NullEvt) class_id 2immediate prio 255 cost 10 e91) queued 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 89 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 91 epoch_requested: 91 NullEvt) class_id 2immediate prio 255 cost 10 e91) pg 0x40f8000 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 88 -> 89 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 88 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 89 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 88 -> 89 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 88 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 89 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 89 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 89 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 89 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0 91 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 91 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 89 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 90 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 89 -> 90 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 89 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 90 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 89 -> 90 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 89 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 89 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 90 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 90 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 90 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 90 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0 91 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 91 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 90 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 91 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 90 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 91 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 90 -> 91 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 90 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 90 -> 91 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 91 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 90 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 89..91 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 51 while current is 91 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0 91 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.369+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 89..91 2026-02-11T23:30:37.369+0000 7fac51168640 10 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 91 epoch_requested: 91 NullEvt 2026-02-11T23:30:37.369+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.369+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 91 2026-02-11T23:30:37.369+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:37.369+0000 7fac55170640 10 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 91 epoch_requested: 91 NullEvt 2026-02-11T23:30:37.369+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:37.369+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.427+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.2 v2:127.0.0.1:6820/2662569638 8 ==== pg_query2(10.0 10.0 query(info 0'0 epoch_sent 92) e92/92) ==== 158+0+0 (unknown 0 0 0) 0x3f3eb40 con 0x3f57c00 2026-02-11T23:30:37.427+0000 7fac71bfd640 15 osd.0 91 enqueue_peering_evt 10.0 epoch_sent: 92 epoch_requested: 92 MQuery 10.0 from 2 query_epoch 92 query: query(info 0'0 epoch_sent 92) 2026-02-11T23:30:37.427+0000 7fac71bfd640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MQuery 10.0 from 2 query_epoch 92 query: query(info 0'0 epoch_sent 92)) class_id 2 prio 255 cost 10 e92) 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MQuery 10.0 from 2 query_epoch 92 query: query(info 0'0 epoch_sent 92)) class_id 2immediate prio 255 cost 10 e92) queued 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MQuery 10.0 from 2 query_epoch 92 query: query(info 0'0 epoch_sent 92)) class_id 2immediate prio 255 cost 10 e92) pg 0 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 map 92 > 91 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 10.0 peering, item epoch is 92, will wait on OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MQuery 10.0 from 2 query_epoch 92 query: query(info 0'0 epoch_sent 92)) class_id 2immediate prio 255 cost 10 e92) 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.427+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.427+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.2 v2:127.0.0.1:6820/2662569638 9 ==== osd_map(89..92 src has 1..92) ==== 2042+0+0 (unknown 0 0 0) 0x3940fc0 con 0x3f57c00 2026-02-11T23:30:37.427+0000 7fac606e4640 20 osd.0 91 OSD::ms_dispatch: osd_map(89..92 src has 1..92) 2026-02-11T23:30:37.427+0000 7fac606e4640 20 osd.0 91 _dispatch 0x3940fc0 osd_map(89..92 src has 1..92) 2026-02-11T23:30:37.427+0000 7fac606e4640 3 osd.0 91 handle_osd_map epochs [89,92], i have 91, src has [1,92] 2026-02-11T23:30:37.427+0000 7fac606e4640 10 osd.0 91 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:37.427+0000 7fac606e4640 10 osd.0 91 handle_osd_map got inc map for epoch 92 2026-02-11T23:30:37.427+0000 7fac606e4640 20 osd.0 91 got_full_map 92, nothing requested 2026-02-11T23:30:37.427+0000 7fac606e4640 20 osd.0 91 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:37.427+0000 7fac606e4640 20 osd.0 91 handle_osd_map pg_num_history pg_num_history(e92 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools ) 2026-02-11T23:30:37.427+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {92={}} 2026-02-11T23:30:37.427+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:37.427+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e92 maps [1~92] lci=[5,92] tlb=1) 2026-02-11T23:30:37.427+0000 7fac5e983640 10 osd.0 91 _committed_osd_maps 92..92 2026-02-11T23:30:37.427+0000 7fac5e983640 10 osd.0 91 advance to epoch 92 (<= last 92 <= newest_map 92) 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0 92 consume_map version 92 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0 91 await_reserved_maps epoch:92 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0 91 await_reserved_maps done 0 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0 92 consume_map 92 -- publish done 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:10.0slot:0x2581600 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 10.0 has no pg and waiting_for_split 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0 92 identify_splits_and_merges 4.0 e91 to e92 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0 92 identify_splits_and_merges 4.0 e91 to e92 pg_nums {45=1} 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0 92 identify_splits_and_merges 1.0 e91 to e92 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0 92 identify_splits_and_merges 1.0 e91 to e92 pg_nums {24=1} 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0 92 prune_pg_created 2026-02-11T23:30:37.427+0000 7fac5e983640 10 osd.0:0.consume_map 92 (was 91) 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 10.0 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 10.0 pending_peering first epoch 92 <= 92, requeueing 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 10.0 to_process <> waiting <> waiting_peering {92=} 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:37.427+0000 7fac5e983640 10 osd.0:1.consume_map 92 (was 91) 2026-02-11T23:30:37.427+0000 7fac5e983640 10 osd.0:2.consume_map 92 (was 91) 2026-02-11T23:30:37.427+0000 7fac5e983640 10 osd.0:3.consume_map 92 (was 91) 2026-02-11T23:30:37.427+0000 7fac5e983640 10 osd.0:4.consume_map 92 (was 91) 2026-02-11T23:30:37.427+0000 7fac5e983640 10 osd.0:5.consume_map 92 (was 91) 2026-02-11T23:30:37.427+0000 7fac5e983640 10 osd.0:6.consume_map 92 (was 91) 2026-02-11T23:30:37.427+0000 7fac5e983640 10 osd.0:7.consume_map 92 (was 91) 2026-02-11T23:30:37.427+0000 7fac5e983640 10 osd.0 92 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.427+0000 7fac5e983640 15 osd.0 92 enqueue_peering_evt 4.0 epoch_sent: 92 epoch_requested: 92 NullEvt 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MQuery 10.0 from 2 query_epoch 92 query: query(info 0'0 epoch_sent 92)) class_id 2immediate prio 255 cost 10 e92) queued 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 MQuery 10.0 from 2 query_epoch 92 query: query(info 0'0 epoch_sent 92)) class_id 2immediate prio 255 cost 10 e92) pg 0 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2 prio 255 cost 10 e92) 2026-02-11T23:30:37.427+0000 7fac51168640 10 osd.0 92 handle_pg_query_nopg 10.0 2026-02-11T23:30:37.427+0000 7fac51168640 10 osd.0 92 pg 10.0 dne 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 92 get_con_osd_cluster to osd.2 from_epoch 92 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 92 get_nextmap_reserved map_reservations: {92=1} 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 92 release_map epoch: 92 2026-02-11T23:30:37.427+0000 7fac51168640 10 osd.0 92 maybe_share_map: con v2:127.0.0.1:6820/2662569638 map epoch 88 -> 92 (shared) 2026-02-11T23:30:37.427+0000 7fac51168640 10 osd.0 92 send_incremental_map epoch range: (89, 92] to v2:127.0.0.1:6820/2662569638 2026-02-11T23:30:37.427+0000 7fac51168640 10 osd.0 92 add_map_inc_bl 89 252 bytes 2026-02-11T23:30:37.427+0000 7fac51168640 10 osd.0 92 add_map_inc_bl 90 574 bytes 2026-02-11T23:30:37.427+0000 7fac51168640 10 osd.0 92 add_map_inc_bl 91 574 bytes 2026-02-11T23:30:37.427+0000 7fac51168640 10 osd.0 92 add_map_inc_bl 92 574 bytes 2026-02-11T23:30:37.427+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] -- osd_map(89..92 src has 1..92) -- 0x38e21c0 con 0x3f57c00 2026-02-11T23:30:37.427+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6820/2662569638,v1:127.0.0.1:6821/2662569638] -- pg_notify2(10.0 10.0 (query:92 sent:92 10.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e92/92) -- 0x3382300 con 0x3f57c00 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.427+0000 7fac5e983640 15 osd.0 92 enqueue_peering_evt 1.0 epoch_sent: 92 epoch_requested: 92 NullEvt 2026-02-11T23:30:37.427+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.427+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2 prio 255 cost 10 e92) 2026-02-11T23:30:37.427+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2immediate prio 255 cost 10 e92) queued 2026-02-11T23:30:37.427+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.427+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2immediate prio 255 cost 10 e92) pg 0x4856000 2026-02-11T23:30:37.427+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.427+0000 7fac55170640 10 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 92 2026-02-11T23:30:37.428+0000 7fac5e983640 7 osd.0 92 activate_map version 92 2026-02-11T23:30:37.428+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 91 -> 92 2026-02-11T23:30:37.428+0000 7fac55170640 10 osd.0 pg_epoch: 91 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:37.428+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:37.428+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2immediate prio 255 cost 10 e92) queued 2026-02-11T23:30:37.428+0000 7fac55971640 20 osd.0 92 agent_entry empty queue 2026-02-11T23:30:37.428+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:37.428+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 92 epoch_requested: 92 NullEvt) class_id 2immediate prio 255 cost 10 e92) pg 0x40f8000 2026-02-11T23:30:37.428+0000 7fac55170640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.428+0000 7fac55170640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.428+0000 7fac55170640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.428+0000 7fac55170640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 92..92 2026-02-11T23:30:37.428+0000 7fac55170640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.428+0000 7fac55170640 20 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Dirtying info: last_persisted is 51 while current is 92 2026-02-11T23:30:37.428+0000 7fac51168640 10 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 92 2026-02-11T23:30:37.428+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 91 -> 92 2026-02-11T23:30:37.428+0000 7fac51168640 10 osd.0 pg_epoch: 91 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:37.428+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:37.428+0000 7fac51168640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:37.428+0000 7fac51168640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:37.428+0000 7fac55170640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 92 epoch_requested: 92 NullEvt 2026-02-11T23:30:37.428+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:37.428+0000 7fac51168640 20 osd.0 92 advance_pg new pool opts old pool opts 2026-02-11T23:30:37.428+0000 7fac51168640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 92..92 2026-02-11T23:30:37.428+0000 7fac51168640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:37.428+0000 7fac51168640 20 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 92 2026-02-11T23:30:37.428+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:37.428+0000 7fac51168640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 92 epoch_requested: 92 NullEvt 2026-02-11T23:30:37.428+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:37.428+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.428+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.428+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:37.428+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.010+0000 7fac60ee5640 20 osd.0 92 reports for 0 queries 2026-02-11T23:30:38.010+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x389ce00 con 0x396a800 2026-02-11T23:30:38.010+0000 7fac60ee5640 15 osd.0 92 collect_pg_stats 2026-02-11T23:30:38.010+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836487 v 0) -- 0x489db00 con 0x396a800 2026-02-11T23:30:38.012+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.2 v2:127.0.0.1:6820/2662569638 10 ==== osd pg remove(epoch 93; pg10.0; ) ==== 32+0+0 (unknown 0 0 0) 0x25d94a0 con 0x3f57c00 2026-02-11T23:30:38.012+0000 7fac71bfd640 7 osd.0 92 handle_fast_pg_remove osd pg remove(epoch 93; pg10.0; ) from osd.2 2026-02-11T23:30:38.012+0000 7fac71bfd640 15 osd.0 92 enqueue_peering_evt 10.0 epoch_sent: 93 epoch_requested: 93 DeleteStart 2026-02-11T23:30:38.012+0000 7fac71bfd640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 DeleteStart) class_id 2 prio 255 cost 10 e93) 2026-02-11T23:30:38.012+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:38.012+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 DeleteStart) class_id 2immediate prio 255 cost 10 e93) queued 2026-02-11T23:30:38.012+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:38.012+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 DeleteStart) class_id 2immediate prio 255 cost 10 e93) pg 0 2026-02-11T23:30:38.012+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 map 93 > 92 2026-02-11T23:30:38.012+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 10.0 peering, item epoch is 93, will wait on OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 DeleteStart) class_id 2immediate prio 255 cost 10 e93) 2026-02-11T23:30:38.012+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.012+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.052+0000 7fac68beb640 10 osd.0 92 tick 2026-02-11T23:30:38.053+0000 7fac68beb640 20 osd.0 92 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:38.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:38.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:38.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:08.169025+0000) 2026-02-11T23:30:38.340+0000 7fac683ea640 10 osd.0 92 tick_without_osd_lock 2026-02-11T23:30:38.340+0000 7fac683ea640 0 osd.0 92 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:38.340+0000 7fac683ea640 20 osd.0 92 tick_without_osd_lock max_waiting_epoch 93, requesting new map 2026-02-11T23:30:38.340+0000 7fac683ea640 10 monclient: _renew_subs 2026-02-11T23:30:38.340+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:38.340+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=93}) -- 0x3e56000 con 0x396bc00 2026-02-11T23:30:38.340+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:38.340+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:38.340+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:38.340+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.129 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:38.340+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:38.341, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:38.340+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:38.340+0000 7fac683ea640 10 osd.0 92 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:38.340+0000 7fac683ea640 20 osd.0 92 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:38.340+0000 7fac683ea640 10 osd.0 92 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:38.340+0000 7fac683ea640 20 osd.0 92 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:38.340+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 49 ==== osd_map(93..94 src has 1..94) ==== 1208+0+0 (crc 0 0 0) 0x46ea8c0 con 0x396bc00 2026-02-11T23:30:38.340+0000 7fac626e8640 20 osd.0 92 OSD::ms_dispatch: osd_map(93..94 src has 1..94) 2026-02-11T23:30:38.340+0000 7fac626e8640 20 osd.0 92 _dispatch 0x46ea8c0 osd_map(93..94 src has 1..94) 2026-02-11T23:30:38.340+0000 7fac626e8640 3 osd.0 92 handle_osd_map epochs [93,94], i have 92, src has [1,94] 2026-02-11T23:30:38.340+0000 7fac626e8640 10 osd.0 92 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:38.340+0000 7fac626e8640 10 osd.0 92 handle_osd_map got inc map for epoch 93 2026-02-11T23:30:38.340+0000 7fac626e8640 20 osd.0 92 got_full_map 93, nothing requested 2026-02-11T23:30:38.340+0000 7fac626e8640 10 osd.0 92 handle_osd_map got inc map for epoch 94 2026-02-11T23:30:38.340+0000 7fac626e8640 20 osd.0 92 got_full_map 94, nothing requested 2026-02-11T23:30:38.340+0000 7fac626e8640 20 osd.0 92 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:38.340+0000 7fac626e8640 20 osd.0 92 handle_osd_map pg_num_history pg_num_history(e94 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools ) 2026-02-11T23:30:38.340+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {93={},94={}} 2026-02-11T23:30:38.340+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:38.340+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e94 maps [1~94] lci=[5,94] tlb=1) 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0 92 _committed_osd_maps 93..94 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0 92 advance to epoch 93 (<= last 94 <= newest_map 94) 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0 93 advance to epoch 94 (<= last 94 <= newest_map 94) 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0 94 consume_map version 94 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0 92 await_reserved_maps epoch:94 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0 92 await_reserved_maps done 0 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0 94 consume_map 94 -- publish done 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:10.0slot:0x2581600 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 10.0 has no pg and waiting_for_split 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0 94 identify_splits_and_merges 4.0 e92 to e94 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0 94 identify_splits_and_merges 4.0 e92 to e94 pg_nums {45=1} 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0 94 identify_splits_and_merges 1.0 e92 to e94 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0 94 identify_splits_and_merges 1.0 e92 to e94 pg_nums {24=1} 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0 94 prune_pg_created 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0:0.consume_map 94 (was 92) 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 10.0 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 10.0 pending_peering first epoch 93 <= 94, requeueing 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 10.0 to_process <> waiting <> waiting_peering {93=} 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0:1.consume_map 94 (was 92) 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0:2.consume_map 94 (was 92) 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0:3.consume_map 94 (was 92) 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0:4.consume_map 94 (was 92) 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0:5.consume_map 94 (was 92) 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0:6.consume_map 94 (was 92) 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0:7.consume_map 94 (was 92) 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 DeleteStart) class_id 2immediate prio 255 cost 10 e93) queued 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 DeleteStart) class_id 2immediate prio 255 cost 10 e93) pg 0 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 op_wq(0) _process 10.0 no pg, peering, doesn't map here e94, discarding OpSchedulerItem(10.0 PGPeeringEvent(epoch_sent: 93 epoch_requested: 93 DeleteStart) class_id 2immediate prio 255 cost 10 e93) 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.340+0000 7fac5e983640 10 osd.0 94 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:38.340+0000 7fac5e983640 15 osd.0 94 enqueue_peering_evt 4.0 epoch_sent: 94 epoch_requested: 94 NullEvt 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 NullEvt) class_id 2 prio 255 cost 10 e94) 2026-02-11T23:30:38.340+0000 7fac5e983640 15 osd.0 94 enqueue_peering_evt 1.0 epoch_sent: 94 epoch_requested: 94 NullEvt 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:38.340+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 NullEvt) class_id 2 prio 255 cost 10 e94) 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 NullEvt) class_id 2immediate prio 255 cost 10 e94) queued 2026-02-11T23:30:38.340+0000 7fac5e983640 7 osd.0 94 activate_map version 94 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 NullEvt) class_id 2immediate prio 255 cost 10 e94) pg 0x4856000 2026-02-11T23:30:38.340+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:38.340+0000 7fac55971640 20 osd.0 94 agent_entry empty queue 2026-02-11T23:30:38.340+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 NullEvt) class_id 2immediate prio 255 cost 10 e94) queued 2026-02-11T23:30:38.340+0000 7fac51168640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 93 2026-02-11T23:30:38.340+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:38.340+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 NullEvt) class_id 2immediate prio 255 cost 10 e94) pg 0x40f8000 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 92 -> 93 2026-02-11T23:30:38.340+0000 7fac51168640 10 osd.0 pg_epoch: 92 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:38.340+0000 7fac51168640 10 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:38.340+0000 7fac51168640 10 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:38.340+0000 7fac55170640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 93 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 94 advance_pg new pool opts old pool opts 2026-02-11T23:30:38.340+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 92 -> 93 2026-02-11T23:30:38.340+0000 7fac55170640 10 osd.0 pg_epoch: 92 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:38.340+0000 7fac51168640 10 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 94 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 93 -> 94 2026-02-11T23:30:38.340+0000 7fac51168640 10 osd.0 pg_epoch: 93 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:38.340+0000 7fac55170640 10 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:38.340+0000 7fac51168640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:38.340+0000 7fac51168640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 94 advance_pg new pool opts old pool opts 2026-02-11T23:30:38.340+0000 7fac55170640 10 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:38.340+0000 7fac51168640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 93..94 2026-02-11T23:30:38.340+0000 7fac51168640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 92 while current is 94 2026-02-11T23:30:38.340+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:38.340+0000 7fac51168640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 94 epoch_requested: 94 NullEvt 2026-02-11T23:30:38.340+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:38.340+0000 7fac55170640 20 osd.0 94 advance_pg new pool opts old pool opts 2026-02-11T23:30:38.340+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.340+0000 7fac55170640 10 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 94 2026-02-11T23:30:38.340+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 93 -> 94 2026-02-11T23:30:38.340+0000 7fac55170640 10 osd.0 pg_epoch: 93 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:38.340+0000 7fac55170640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:38.340+0000 7fac55170640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:38.340+0000 7fac55170640 20 osd.0 94 advance_pg new pool opts old pool opts 2026-02-11T23:30:38.340+0000 7fac55170640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 93..94 2026-02-11T23:30:38.340+0000 7fac55170640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:38.340+0000 7fac55170640 20 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 94 2026-02-11T23:30:38.340+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:38.340+0000 7fac55170640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 94 epoch_requested: 94 NullEvt 2026-02-11T23:30:38.340+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:38.340+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:38.379+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 7 ==== osd_ping(ping e91 up_from 17 ping_stamp 2026-02-11T23:30:38.380352+0000/23.904388428s send_stamp 23.904388428s delta_ub -7.310208797s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a180 con 0x3f56800 2026-02-11T23:30:38.379+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 7 ==== osd_ping(ping e91 up_from 17 ping_stamp 2026-02-11T23:30:38.380352+0000/23.904388428s send_stamp 23.904388428s delta_ub -7.310208797s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a780 con 0x3f56c00 2026-02-11T23:30:38.379+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310467720s,-7.310208797s]) 2026-02-11T23:30:38.379+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:38.380352+0000/23.904388428s send_stamp 31.214857101s delta_ub 7.310467720s) -- 0x3d0a480 con 0x3f56800 2026-02-11T23:30:38.379+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:38.379+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:38.379+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:38.379+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 88 to ping map epoch of 91 2026-02-11T23:30:38.379+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 map epoch 91 -> 94 (shared) 2026-02-11T23:30:38.379+0000 7fac71bfd640 10 osd.0 94 send_incremental_map epoch range: (92, 94] to v2:127.0.0.1:6828/1114810440 2026-02-11T23:30:38.379+0000 7fac71bfd640 10 osd.0 94 add_map_inc_bl 93 582 bytes 2026-02-11T23:30:38.379+0000 7fac71bfd640 10 osd.0 94 add_map_inc_bl 94 574 bytes 2026-02-11T23:30:38.379+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6828/1114810440,v1:127.0.0.1:6829/1114810440] -- osd_map(92..94 src has 1..94) -- 0x38e21c0 con 0x3741000 2026-02-11T23:30:38.379+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310637474s,-7.310208797s]) 2026-02-11T23:30:38.379+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:38.380352+0000/23.904388428s send_stamp 31.215026855s delta_ub 7.310637474s) -- 0x3d0a900 con 0x3f56c00 2026-02-11T23:30:38.379+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:38.379+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:38.379+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:38.379+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:38.956+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 9 ==== osd_ping(ping e94 up_from 21 ping_stamp 2026-02-11T23:30:38.957065+0000/21.904571533s send_stamp 21.904571533s delta_ub -9.886527061s) ==== 2033+0+0 (crc 0 0 0) 0x3d0aa80 con 0x3f57400 2026-02-11T23:30:38.956+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886984825s,-9.886527061s]) 2026-02-11T23:30:38.956+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:38.957065+0000/21.904571533s send_stamp 31.791557312s delta_ub 9.886984825s) -- 0x3d0a000 con 0x3f57400 2026-02-11T23:30:38.956+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:38.956+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 9 ==== osd_ping(ping e94 up_from 21 ping_stamp 2026-02-11T23:30:38.957065+0000/21.904571533s send_stamp 21.904571533s delta_ub -9.886527061s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a480 con 0x3f57800 2026-02-11T23:30:38.956+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:38.956+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:38.956+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 88 to ping map epoch of 94 2026-02-11T23:30:38.956+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:38.956+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887020111s,-9.886527061s]) 2026-02-11T23:30:38.956+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:38.957065+0000/21.904571533s send_stamp 31.791593552s delta_ub 9.887020111s) -- 0x3d0a180 con 0x3f57800 2026-02-11T23:30:38.956+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:38.956+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:38.956+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:38.956+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:39.063+0000 7fac68beb640 10 osd.0 94 tick 2026-02-11T23:30:39.063+0000 7fac68beb640 20 osd.0 94 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:39.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:39.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:39.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:09.169049+0000) 2026-02-11T23:30:39.298+0000 7fac683ea640 10 osd.0 94 tick_without_osd_lock 2026-02-11T23:30:39.298+0000 7fac683ea640 0 osd.0 94 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:39.298+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:39.298+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.129 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:39.298+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:39.299, recovery is active?:false restrictions:<....> 2026-02-11T23:30:39.298+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:39.298+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:39.298+0000 7fac683ea640 20 osd.0 94 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:39.298+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:39.298+0000 7fac683ea640 20 osd.0 94 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:39.769+0000 7fac4d160640 5 osd.0 94 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x6ccce/0xf0000, compress 0x0/0x0/0x0, omap 0x9236, meta 0x1a46dca), peers [1,2,3,4] op hist []) 2026-02-11T23:30:39.769+0000 7fac4d160640 20 osd.0 94 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:39.769+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 32.604743958s delta_ub 1.981085181s) -- 0x3d0ac00 con 0x396b400 2026-02-11T23:30:39.769+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 32.604743958s delta_ub 1.981085181s) -- 0x3d0ad80 con 0x3752400 2026-02-11T23:30:39.769+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 32.604743958s delta_ub 4.649088860s) -- 0x3d0af00 con 0x396b800 2026-02-11T23:30:39.769+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 32.604743958s delta_ub 4.649088860s) -- 0x3d0b080 con 0x34c1000 2026-02-11T23:30:39.769+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 32.604743958s delta_ub 7.310637474s) -- 0x3d0b200 con 0x3740000 2026-02-11T23:30:39.769+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 32.604743958s delta_ub 7.310637474s) -- 0x3d0b380 con 0x3740400 2026-02-11T23:30:39.769+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 32.604743958s delta_ub 9.887020111s) -- 0x3d0b500 con 0x3740800 2026-02-11T23:30:39.769+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 32.604743958s delta_ub 9.887020111s) -- 0x3d0b680 con 0x3740c00 2026-02-11T23:30:39.769+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 9 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 30.623964310s delta_ub -1.980781078s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b200 con 0x396b400 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:39.769+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 88 to ping map epoch of 94 2026-02-11T23:30:39.769+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980928659s,-1.980781078s]) 2026-02-11T23:30:39.769+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 8 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 27.955842972s delta_ub -4.648904324s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b500 con 0x396b800 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:39.769+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 92 to ping map epoch of 94 2026-02-11T23:30:39.769+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:39.769+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 8 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 27.955865860s delta_ub -4.648880005s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b080 con 0x34c1000 2026-02-11T23:30:39.769+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 6 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 25.294559479s delta_ub -7.310187340s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b680 con 0x3740000 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649099350s,-4.648904324s]) 2026-02-11T23:30:39.769+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 6 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 25.294506073s delta_ub -7.310239792s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b500 con 0x3740400 2026-02-11T23:30:39.769+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:39.769+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:39.769+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:39.769+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:39.769+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649157524s,-4.648880005s]) 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:39.769+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310501099s,-7.310187340s]) 2026-02-11T23:30:39.769+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 9 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 30.623998642s delta_ub -1.980747938s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b080 con 0x3752400 2026-02-11T23:30:39.769+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 6 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 22.718069077s delta_ub -9.886677742s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b680 con 0x3740c00 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:39.769+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310564995s,-7.310239792s]) 2026-02-11T23:30:39.769+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:39.769+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:39.769+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 6 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:39.770331+0000/32.604743958s send_stamp 22.717996597s delta_ub -9.886748314s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b500 con 0x3740800 2026-02-11T23:30:39.769+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:39.769+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:39.769+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.981099486s,-1.980747938s]) 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:39.769+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:39.769+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887050629s,-9.886677742s]) 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:39.769+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:39.769+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887133598s,-9.886748314s]) 2026-02-11T23:30:40.033+0000 7fac68beb640 10 osd.0 94 tick 2026-02-11T23:30:40.033+0000 7fac68beb640 20 osd.0 94 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:40.079+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 8 ==== osd_ping(ping e94 up_from 17 ping_stamp 2026-02-11T23:30:40.080610+0000/25.604646683s send_stamp 25.604646683s delta_ub -7.310187340s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b080 con 0x3f56800 2026-02-11T23:30:40.079+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 8 ==== osd_ping(ping e94 up_from 17 ping_stamp 2026-02-11T23:30:40.080610+0000/25.604646683s send_stamp 25.604646683s delta_ub -7.310187340s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b500 con 0x3f56c00 2026-02-11T23:30:40.079+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310453415s,-7.310187340s]) 2026-02-11T23:30:40.079+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:40.080610+0000/25.604646683s send_stamp 32.915100098s delta_ub 7.310453415s) -- 0x3d0ad80 con 0x3f56800 2026-02-11T23:30:40.079+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:40.079+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:40.079+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:40.079+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:40.079+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310492992s,-7.310187340s]) 2026-02-11T23:30:40.079+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:40.080610+0000/25.604646683s send_stamp 32.915138245s delta_ub 7.310492992s) -- 0x3d0b380 con 0x3f56c00 2026-02-11T23:30:40.079+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:40.079+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:40.079+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:40.079+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:40.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:40.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:40.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:10.169086+0000) 2026-02-11T23:30:40.168+0000 7fac616e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:40.257+0000 7fac683ea640 10 osd.0 94 tick_without_osd_lock 2026-02-11T23:30:40.257+0000 7fac683ea640 0 osd.0 94 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:40.257+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:40.257+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:40.257+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:40.257+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.119 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:40.257+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:40.258, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:40.257+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:40.257+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:40.257+0000 7fac683ea640 20 osd.0 94 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:40.257+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:40.257+0000 7fac683ea640 20 osd.0 94 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:40.617+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 6 ==== osd_ping(ping e94 up_from 13 ping_stamp 2026-02-11T23:30:40.618240+0000/28.803634644s send_stamp 28.803634644s delta_ub -4.648880005s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b380 con 0x3f56000 2026-02-11T23:30:40.617+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649081230s,-4.648880005s]) 2026-02-11T23:30:40.617+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:40.618240+0000/28.803634644s send_stamp 33.452716827s delta_ub 4.649081230s) -- 0x3d0b500 con 0x3f56000 2026-02-11T23:30:40.617+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:40.617+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:40.617+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:40.617+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:40.617+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 6 ==== osd_ping(ping e94 up_from 13 ping_stamp 2026-02-11T23:30:40.618240+0000/28.803634644s send_stamp 28.803634644s delta_ub -4.648880005s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b680 con 0x3f56400 2026-02-11T23:30:40.617+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649093628s,-4.648880005s]) 2026-02-11T23:30:40.617+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:40.618240+0000/28.803634644s send_stamp 33.452732086s delta_ub 4.649093628s) -- 0x3d0b200 con 0x3f56400 2026-02-11T23:30:40.617+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:40.617+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:40.617+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:40.617+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:41.006+0000 7fac68beb640 10 osd.0 94 tick 2026-02-11T23:30:41.006+0000 7fac68beb640 20 osd.0 94 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:41.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:41.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:41.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:11.169171+0000) 2026-02-11T23:30:41.216+0000 7fac683ea640 10 osd.0 94 tick_without_osd_lock 2026-02-11T23:30:41.216+0000 7fac683ea640 0 osd.0 94 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:41.216+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:41.216+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:41.216+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:41.216+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.119 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:41.216+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:41.217, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:41.216+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:41.216+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:41.216+0000 7fac683ea640 20 osd.0 94 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:41.216+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:41.216+0000 7fac683ea640 20 osd.0 94 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:42.008+0000 7fac68beb640 10 osd.0 94 tick 2026-02-11T23:30:42.008+0000 7fac68beb640 20 osd.0 94 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:42.069+0000 7fac4d160640 5 osd.0 94 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x6ccce/0xf0000, compress 0x0/0x0/0x0, omap 0x9236, meta 0x1a46dca), peers [1,2,3,4] op hist []) 2026-02-11T23:30:42.069+0000 7fac4d160640 20 osd.0 94 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:42.069+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 34.904945374s delta_ub 1.981099486s) -- 0x3d0b800 con 0x396b400 2026-02-11T23:30:42.069+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 34.904945374s delta_ub 1.981099486s) -- 0x3d0b980 con 0x3752400 2026-02-11T23:30:42.069+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 34.904945374s delta_ub 4.649093628s) -- 0x3d0bb00 con 0x396b800 2026-02-11T23:30:42.069+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 34.904945374s delta_ub 4.649093628s) -- 0x3d0bc80 con 0x34c1000 2026-02-11T23:30:42.069+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 34.904945374s delta_ub 7.310492992s) -- 0x3d0be00 con 0x3740000 2026-02-11T23:30:42.069+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 34.904945374s delta_ub 7.310492992s) -- 0x3d0a600 con 0x3740400 2026-02-11T23:30:42.069+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 34.904945374s delta_ub 9.887133598s) -- 0x3d0aa80 con 0x3740800 2026-02-11T23:30:42.069+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 34.904945374s delta_ub 9.887133598s) -- 0x3d0a000 con 0x3740c00 2026-02-11T23:30:42.069+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 10 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 32.924156189s delta_ub -1.980790854s) ==== 2033+0+0 (crc 0 0 0) 0x3d0be00 con 0x396b400 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.069+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980894089s,-1.980790854s]) 2026-02-11T23:30:42.069+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 7 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 27.594659805s delta_ub -7.310286045s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a000 con 0x3740000 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.069+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310416222s,-7.310286045s]) 2026-02-11T23:30:42.069+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 9 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 30.256011963s delta_ub -4.648935795s) ==== 2033+0+0 (crc 0 0 0) 0x3d0aa80 con 0x396b800 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.069+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 10 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 32.924179077s delta_ub -1.980766773s) ==== 2033+0+0 (crc 0 0 0) 0x3d0bc80 con 0x3752400 2026-02-11T23:30:42.069+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649086475s,-4.648935795s]) 2026-02-11T23:30:42.069+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:42.069+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 7 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 25.018182755s delta_ub -9.886761665s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a000 con 0x3740c00 2026-02-11T23:30:42.069+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.069+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.069+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 7 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 27.594676971s delta_ub -7.310271263s) ==== 2033+0+0 (crc 0 0 0) 0x3d0aa80 con 0x3740400 2026-02-11T23:30:42.069+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.069+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980939627s,-1.980766773s]) 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.069+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.069+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886964798s,-9.886761665s]) 2026-02-11T23:30:42.069+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 9 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 30.256042480s delta_ub -4.648905277s) ==== 2033+0+0 (crc 0 0 0) 0x3d0bc80 con 0x34c1000 2026-02-11T23:30:42.069+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:42.069+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.069+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.069+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.069+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649118900s,-4.648905277s]) 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.069+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310499668s,-7.310271263s]) 2026-02-11T23:30:42.069+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 7 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:42.070531+0000/34.904945374s send_stamp 25.018157959s delta_ub -9.886789322s) ==== 2033+0+0 (crc 0 0 0) 0x3d0aa80 con 0x3740800 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.069+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.069+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887037277s,-9.886789322s]) 2026-02-11T23:30:42.156+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 30 ==== pg_lease(4.0 pg_lease(ru 33.104816437s ub 41.104904175s int 16.000000000s) e94/94) ==== 58+0+0 (unknown 0 0 0) 0x3d0a000 con 0x3e4a000 2026-02-11T23:30:42.156+0000 7fac703fa640 15 osd.0 94 enqueue_peering_evt 4.0 epoch_sent: 94 epoch_requested: 94 MLease epoch 94 from osd.4 pg_lease(ru 33.104816437s ub 41.104904175s int 16.000000000s) 2026-02-11T23:30:42.156+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 MLease epoch 94 from osd.4 pg_lease(ru 33.104816437s ub 41.104904175s int 16.000000000s)) class_id 2 prio 255 cost 10 e94) 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 MLease epoch 94 from osd.4 pg_lease(ru 33.104816437s ub 41.104904175s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e94) queued 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 MLease epoch 94 from osd.4 pg_lease(ru 33.104816437s ub 41.104904175s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e94) pg 0x4856000 2026-02-11T23:30:42.156+0000 7fac51168640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 94 epoch_requested: 94 MLease epoch 94 from osd.4 pg_lease(ru 33.104816437s ub 41.104904175s int 16.000000000s) 2026-02-11T23:30:42.156+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:42.156+0000 7fac51168640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 33.104816437s ub 41.104904175s int 16.000000000s) 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.886789322s -> ru 42.991607666s 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 42.991607666s 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.887037277s -> ruub 50.991943359s 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 50.991943359s 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.156+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_lease_ack(4.0 pg_lease_ack(ruub 41.104904175s) e94/94) -- 0x2581e40 con 0x3e4a000 2026-02-11T23:30:42.156+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:42.156+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:42.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:42.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:42.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:12.169198+0000) 2026-02-11T23:30:42.215+0000 7fac683ea640 10 osd.0 94 tick_without_osd_lock 2026-02-11T23:30:42.215+0000 7fac683ea640 0 osd.0 94 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:42.215+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:42.215+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:42.215+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:42.215+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.119 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:42.215+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:42.216, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:42.215+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:42.215+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:42.215+0000 7fac683ea640 20 osd.0 94 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:42.215+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:42.215+0000 7fac683ea640 20 osd.0 94 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:42.349+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 10 ==== osd_ping(ping e94 up_from 9 ping_stamp 2026-02-11T23:30:42.350563+0000/33.204128265s send_stamp 33.204128265s delta_ub -1.980766773s) ==== 2033+0+0 (crc 0 0 0) 0x3d0aa80 con 0x396a000 2026-02-11T23:30:42.349+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980919600s,-1.980766773s]) 2026-02-11T23:30:42.349+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 10 ==== osd_ping(ping e94 up_from 9 ping_stamp 2026-02-11T23:30:42.350563+0000/33.204128265s send_stamp 33.204128265s delta_ub -1.980766773s) ==== 2033+0+0 (crc 0 0 0) 0x3d0bc80 con 0x396a400 2026-02-11T23:30:42.349+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:42.350563+0000/33.204128265s send_stamp 35.185047150s delta_ub 1.980919600s) -- 0x3d0a600 con 0x396a000 2026-02-11T23:30:42.349+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:42.349+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.349+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.349+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.349+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980936289s,-1.980766773s]) 2026-02-11T23:30:42.349+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:42.350563+0000/33.204128265s send_stamp 35.185062408s delta_ub 1.980936289s) -- 0x3b1b980 con 0x396a400 2026-02-11T23:30:42.349+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:42.349+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.349+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.349+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.569+0000 7fac4d160640 5 osd.0 94 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x6ccce/0xf0000, compress 0x0/0x0/0x0, omap 0x9236, meta 0x1a46dca), peers [1,2,3,4] op hist []) 2026-02-11T23:30:42.569+0000 7fac4d160640 20 osd.0 94 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:42.569+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 35.405120850s delta_ub 1.980936289s) -- 0x3b1a780 con 0x396b400 2026-02-11T23:30:42.569+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 35.405120850s delta_ub 1.980936289s) -- 0x3b1bc80 con 0x3752400 2026-02-11T23:30:42.569+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 35.405120850s delta_ub 4.649118900s) -- 0x3b1b680 con 0x396b800 2026-02-11T23:30:42.569+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 35.405120850s delta_ub 4.649118900s) -- 0x3b1a300 con 0x34c1000 2026-02-11T23:30:42.569+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 35.405120850s delta_ub 7.310499668s) -- 0x3b1b080 con 0x3740000 2026-02-11T23:30:42.569+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 35.405120850s delta_ub 7.310499668s) -- 0x3b1b800 con 0x3740400 2026-02-11T23:30:42.569+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 35.405120850s delta_ub 9.887037277s) -- 0x3b1a480 con 0x3740800 2026-02-11T23:30:42.569+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 35.405120850s delta_ub 9.887037277s) -- 0x3919980 con 0x3740c00 2026-02-11T23:30:42.569+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 11 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 33.424316406s delta_ub -1.980801225s) ==== 2033+0+0 (crc 0 0 0) 0x3919980 con 0x396b400 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.569+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980883002s,-1.980801225s]) 2026-02-11T23:30:42.569+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 8 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 25.518276215s delta_ub -9.886841774s) ==== 2033+0+0 (crc 0 0 0) 0x3919980 con 0x3740c00 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.569+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886936188s,-9.886841774s]) 2026-02-11T23:30:42.569+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 8 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 28.094802856s delta_ub -7.310316086s) ==== 2033+0+0 (crc 0 0 0) 0x3919980 con 0x3740000 2026-02-11T23:30:42.569+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 10 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 30.756166458s delta_ub -4.648952961s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a480 con 0x396b800 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.569+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.569+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310426235s,-7.310316086s]) 2026-02-11T23:30:42.569+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 11 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 33.424327850s delta_ub -1.980791092s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a300 con 0x3752400 2026-02-11T23:30:42.569+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:42.569+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.569+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.569+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.569+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980905414s,-1.980791092s]) 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.569+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649079323s,-4.648952961s]) 2026-02-11T23:30:42.569+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 10 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 30.756181717s delta_ub -4.648936272s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a300 con 0x34c1000 2026-02-11T23:30:42.569+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:42.569+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.569+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.569+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.569+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 8 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 28.094816208s delta_ub -7.310302258s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a480 con 0x3740400 2026-02-11T23:30:42.569+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649071693s,-4.648936272s]) 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.569+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310447693s,-7.310302258s]) 2026-02-11T23:30:42.569+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 8 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:42.570702+0000/35.405120850s send_stamp 25.518299103s delta_ub -9.886820793s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a480 con 0x3740800 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.569+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.569+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886980057s,-9.886820793s]) 2026-02-11T23:30:42.849+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 11 ==== osd_ping(ping e94 up_from 9 ping_stamp 2026-02-11T23:30:42.850770+0000/33.704334259s send_stamp 33.704334259s delta_ub -1.980791092s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a300 con 0x396a400 2026-02-11T23:30:42.849+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980929971s,-1.980791092s]) 2026-02-11T23:30:42.849+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:42.850770+0000/33.704334259s send_stamp 35.685264587s delta_ub 1.980929971s) -- 0x3b1b500 con 0x396a400 2026-02-11T23:30:42.849+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:42.849+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.849+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.849+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.849+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 11 ==== osd_ping(ping e94 up_from 9 ping_stamp 2026-02-11T23:30:42.850770+0000/33.704334259s send_stamp 33.704334259s delta_ub -1.980791092s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a480 con 0x396a000 2026-02-11T23:30:42.849+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980949044s,-1.980791092s]) 2026-02-11T23:30:42.849+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:42.850770+0000/33.704334259s send_stamp 35.685283661s delta_ub 1.980949044s) -- 0x3b1b800 con 0x396a000 2026-02-11T23:30:42.849+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:42.849+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:42.849+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:42.849+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:42.900+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> conn(0x48e6000 0x40eb600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:42.900+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> conn(0x48e6000 0x40eb600 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:42.900+0000 7fac71bfd640 10 osd.0 94 ms_handle_fast_authentication new session 0x3941a40 con 0x48e6000 entity client.admin addr 2026-02-11T23:30:42.900+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/584251372 conn(0x48e6000 0x40eb600 crc :-1 s=READY pgs=3 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=client.4237 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:42.900+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.4237 127.0.0.1:0/584251372 1 ==== command(tid 1: {"prefix": "get_command_descriptions"}) ==== 62+0+0 (crc 0 0 0) 0x25d94a0 con 0x48e6000 2026-02-11T23:30:42.900+0000 7fac626e8640 20 osd.0 94 OSD::ms_dispatch: command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:30:42.900+0000 7fac626e8640 20 osd.0 94 _dispatch 0x25d94a0 command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:30:42.901+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/584251372 -- command_reply(tid 1: 0 ) -- 0x25d9340 con 0x48e6000 2026-02-11T23:30:42.909+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.4237 127.0.0.1:0/584251372 2 ==== command(tid 2: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x25d9340 con 0x48e6000 2026-02-11T23:30:42.909+0000 7fac626e8640 20 osd.0 94 OSD::ms_dispatch: command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:42.909+0000 7fac626e8640 20 osd.0 94 _dispatch 0x25d9340 command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:42.909+0000 7fac713fc640 15 osd.0 94 collect_pg_stats 2026-02-11T23:30:42.909+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836488 v 0) -- 0x4103500 con 0x396a800 2026-02-11T23:30:42.909+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/584251372 -- command_reply(tid 2: 0 ) -- 0x25d94a0 con 0x48e6000 2026-02-11T23:30:42.914+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.4237 127.0.0.1:0/584251372 3 ==== command(tid 3: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x25d94a0 con 0x48e6000 2026-02-11T23:30:42.914+0000 7fac626e8640 20 osd.0 94 OSD::ms_dispatch: command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:42.914+0000 7fac626e8640 20 osd.0 94 _dispatch 0x25d94a0 command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:42.914+0000 7fac713fc640 15 osd.0 94 collect_pg_stats 2026-02-11T23:30:42.914+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836489 v 0) -- 0x3d9e000 con 0x396a800 2026-02-11T23:30:42.914+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/584251372 -- command_reply(tid 3: 0 ) -- 0x25d9340 con 0x48e6000 2026-02-11T23:30:42.914+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/584251372 conn(0x48e6000 msgr2=0x40eb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 79 2026-02-11T23:30:42.914+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/584251372 conn(0x48e6000 msgr2=0x40eb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-11T23:30:42.914+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/584251372 conn(0x48e6000 0x40eb600 crc :-1 s=READY pgs=3 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-11T23:30:42.914+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/584251372 conn(0x48e6000 0x40eb600 crc :-1 s=READY pgs=3 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:42.914+0000 7fac626e8640 2 osd.0 94 ms_handle_reset con 0x48e6000 session 0x3941a40 2026-02-11T23:30:43.010+0000 7fac60ee5640 20 osd.0 94 reports for 0 queries 2026-02-11T23:30:43.010+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x34c3c00 con 0x396a800 2026-02-11T23:30:43.010+0000 7fac60ee5640 15 osd.0 94 collect_pg_stats 2026-02-11T23:30:43.010+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836490 v 0) -- 0x3d9e600 con 0x396a800 2026-02-11T23:30:43.025+0000 7fac68beb640 10 osd.0 94 tick 2026-02-11T23:30:43.025+0000 7fac68beb640 20 osd.0 94 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:43.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:43.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:43.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:13.169264+0000) 2026-02-11T23:30:43.194+0000 7fac683ea640 10 osd.0 94 tick_without_osd_lock 2026-02-11T23:30:43.194+0000 7fac683ea640 0 osd.0 94 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:43.194+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:43.194+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:43.194+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:43.194+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.119 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:43.194+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:43.196, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:43.194+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:43.194+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:43.194+0000 7fac683ea640 20 osd.0 94 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:43.194+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:43.194+0000 7fac683ea640 20 osd.0 94 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:43.656+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 10 ==== osd_ping(ping e94 up_from 21 ping_stamp 2026-02-11T23:30:43.657342+0000/26.604848862s send_stamp 26.604848862s delta_ub -9.886820793s) ==== 2033+0+0 (crc 0 0 0) 0x3919980 con 0x3f57400 2026-02-11T23:30:43.656+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 10 ==== osd_ping(ping e94 up_from 21 ping_stamp 2026-02-11T23:30:43.657342+0000/26.604848862s send_stamp 26.604848862s delta_ub -9.886820793s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b500 con 0x3f57800 2026-02-11T23:30:43.656+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887070656s,-9.886820793s]) 2026-02-11T23:30:43.656+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:43.657342+0000/26.604848862s send_stamp 36.491920471s delta_ub 9.887070656s) -- 0x3b1b080 con 0x3f57400 2026-02-11T23:30:43.656+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:43.656+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:43.656+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:43.656+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:43.656+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887113571s,-9.886820793s]) 2026-02-11T23:30:43.656+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:43.657342+0000/26.604848862s send_stamp 36.491958618s delta_ub 9.887113571s) -- 0x3b1a300 con 0x3f57800 2026-02-11T23:30:43.656+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:43.656+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:43.656+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:43.656+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:43.669+0000 7fac4d160640 5 osd.0 94 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x6ccce/0xf0000, compress 0x0/0x0/0x0, omap 0x9236, meta 0x1a46dca), peers [1,2,3,4] op hist []) 2026-02-11T23:30:43.669+0000 7fac4d160640 20 osd.0 94 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:43.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 36.505260468s delta_ub 1.980949044s) -- 0x3919200 con 0x396b400 2026-02-11T23:30:43.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 36.505260468s delta_ub 1.980949044s) -- 0x3d0bc80 con 0x3752400 2026-02-11T23:30:43.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 36.505260468s delta_ub 4.649071693s) -- 0x3d0a480 con 0x396b800 2026-02-11T23:30:43.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 36.505260468s delta_ub 4.649071693s) -- 0x3d0a180 con 0x34c1000 2026-02-11T23:30:43.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 36.505260468s delta_ub 7.310447693s) -- 0x3d0b080 con 0x3740000 2026-02-11T23:30:43.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 36.505260468s delta_ub 7.310447693s) -- 0x3d0ad80 con 0x3740400 2026-02-11T23:30:43.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 36.505260468s delta_ub 9.887113571s) -- 0x3d0b980 con 0x3740800 2026-02-11T23:30:43.669+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 36.505260468s delta_ub 9.887113571s) -- 0x3d0a300 con 0x3740c00 2026-02-11T23:30:43.669+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 12 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 34.524486542s delta_ub -1.980774760s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a300 con 0x396b400 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:43.669+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980904460s,-1.980774760s]) 2026-02-11T23:30:43.669+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 12 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 34.524509430s delta_ub -1.980753541s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a180 con 0x3752400 2026-02-11T23:30:43.669+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 9 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 29.194988251s delta_ub -7.310273170s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a300 con 0x3740000 2026-02-11T23:30:43.669+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:43.669+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:43.669+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:43.669+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:43.669+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980897784s,-1.980753541s]) 2026-02-11T23:30:43.669+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 11 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 31.856353760s delta_ub -4.648906708s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b980 con 0x396b800 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:43.669+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649080276s,-4.648906708s]) 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:43.669+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310456753s,-7.310273170s]) 2026-02-11T23:30:43.669+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 9 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 26.618492126s delta_ub -9.886767387s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a300 con 0x3740c00 2026-02-11T23:30:43.669+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 11 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 31.856380463s delta_ub -4.648880959s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a180 con 0x34c1000 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:43.669+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 9 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 29.195018768s delta_ub -7.310241699s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b980 con 0x3740400 2026-02-11T23:30:43.669+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:43.669+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886978149s,-9.886767387s]) 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:43.669+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310458183s,-7.310241699s]) 2026-02-11T23:30:43.669+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:43.669+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:43.669+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 9 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:43.670845+0000/36.505260468s send_stamp 26.618511200s delta_ub -9.886748314s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b980 con 0x3740800 2026-02-11T23:30:43.669+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:43.669+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:43.669+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649111748s,-4.648880959s]) 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:43.669+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:43.669+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887004852s,-9.886748314s]) 2026-02-11T23:30:44.047+0000 7fac68beb640 10 osd.0 94 tick 2026-02-11T23:30:44.047+0000 7fac68beb640 20 osd.0 94 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:44.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:44.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:44.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:14.169299+0000) 2026-02-11T23:30:44.170+0000 7fac683ea640 10 osd.0 94 tick_without_osd_lock 2026-02-11T23:30:44.170+0000 7fac683ea640 0 osd.0 94 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:44.170+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:44.170+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.119 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:44.170+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:44.171, recovery is active?:false restrictions:<....> 2026-02-11T23:30:44.170+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:44.170+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:44.170+0000 7fac683ea640 20 osd.0 94 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:44.170+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:44.170+0000 7fac683ea640 20 osd.0 94 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:44.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 23 ==== pg_lease(1.0 pg_lease(ru 43.071571350s ub 51.071666718s int 16.000000000s) e94/94) ==== 58+0+0 (unknown 0 0 0) 0x3d0a300 con 0x3752000 2026-02-11T23:30:44.217+0000 7fac703fa640 15 osd.0 94 enqueue_peering_evt 1.0 epoch_sent: 94 epoch_requested: 94 MLease epoch 94 from osd.1 pg_lease(ru 43.071571350s ub 51.071666718s int 16.000000000s) 2026-02-11T23:30:44.217+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 MLease epoch 94 from osd.1 pg_lease(ru 43.071571350s ub 51.071666718s int 16.000000000s)) class_id 2 prio 255 cost 10 e94) 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 MLease epoch 94 from osd.1 pg_lease(ru 43.071571350s ub 51.071666718s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e94) queued 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 94 epoch_requested: 94 MLease epoch 94 from osd.1 pg_lease(ru 43.071571350s ub 51.071666718s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e94) pg 0x40f8000 2026-02-11T23:30:44.217+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:44.217+0000 7fac51168640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 94 epoch_requested: 94 MLease epoch 94 from osd.1 pg_lease(ru 43.071571350s ub 51.071666718s int 16.000000000s) 2026-02-11T23:30:44.217+0000 7fac51168640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 43.071571350s ub 51.071666718s int 16.000000000s) 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -1.980753541s -> ru 45.052322388s 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 45.052322388s 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.980897784s -> ruub 53.052566528s 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 53.052566528s 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:44.217+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_lease_ack(1.0 pg_lease_ack(ruub 51.071666718s) e94/94) -- 0x2581340 con 0x3752000 2026-02-11T23:30:44.217+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:44.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:44.717+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 7 ==== osd_ping(ping e94 up_from 13 ping_stamp 2026-02-11T23:30:44.718432+0000/32.903827667s send_stamp 32.903827667s delta_ub -4.648880959s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a300 con 0x3f56400 2026-02-11T23:30:44.717+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 7 ==== osd_ping(ping e94 up_from 13 ping_stamp 2026-02-11T23:30:44.718432+0000/32.903827667s send_stamp 32.903827667s delta_ub -4.648880959s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b980 con 0x3f56000 2026-02-11T23:30:44.717+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649126053s,-4.648880959s]) 2026-02-11T23:30:44.717+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:44.718432+0000/32.903827667s send_stamp 37.552951813s delta_ub 4.649126053s) -- 0x3d0b080 con 0x3f56400 2026-02-11T23:30:44.717+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:44.717+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:44.717+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:44.717+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:44.717+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649149895s,-4.648880959s]) 2026-02-11T23:30:44.717+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:44.718432+0000/32.903827667s send_stamp 37.552974701s delta_ub 4.649149895s) -- 0x3d0ad80 con 0x3f56000 2026-02-11T23:30:44.717+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:44.717+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:44.717+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:44.717+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.093+0000 7fac68beb640 10 osd.0 94 tick 2026-02-11T23:30:45.093+0000 7fac68beb640 20 osd.0 94 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:45.149+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 12 ==== osd_ping(ping e94 up_from 9 ping_stamp 2026-02-11T23:30:45.150936+0000/36.004501343s send_stamp 36.004501343s delta_ub -1.980753541s) ==== 2033+0+0 (crc 0 0 0) 0x3d0a180 con 0x396a400 2026-02-11T23:30:45.149+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980942965s,-1.980753541s]) 2026-02-11T23:30:45.149+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:45.150936+0000/36.004501343s send_stamp 37.985443115s delta_ub 1.980942965s) -- 0x3d0bc80 con 0x396a400 2026-02-11T23:30:45.149+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 12 ==== osd_ping(ping e94 up_from 9 ping_stamp 2026-02-11T23:30:45.150936+0000/36.004501343s send_stamp 36.004501343s delta_ub -1.980753541s) ==== 2033+0+0 (crc 0 0 0) 0x3d0ad80 con 0x396a000 2026-02-11T23:30:45.149+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:45.149+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.149+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.149+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.149+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980965734s,-1.980753541s]) 2026-02-11T23:30:45.149+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:45.150936+0000/36.004501343s send_stamp 37.985469818s delta_ub 1.980965734s) -- 0x3d0b980 con 0x396a000 2026-02-11T23:30:45.149+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:45.149+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.149+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.149+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:45.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:45.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:15.169366+0000) 2026-02-11T23:30:45.189+0000 7fac683ea640 10 osd.0 94 tick_without_osd_lock 2026-02-11T23:30:45.189+0000 7fac683ea640 0 osd.0 94 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:45.189+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:45.189+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:45.189+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:45.189+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.113 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:45.189+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:45.190, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:45.189+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:45.189+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:45.189+0000 7fac683ea640 20 osd.0 94 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:45.189+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:45.189+0000 7fac683ea640 20 osd.0 94 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:45.969+0000 7fac4d160640 5 osd.0 94 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x6ccce/0xf0000, compress 0x0/0x0/0x0, omap 0x9236, meta 0x1a46dca), peers [1,2,3,4] op hist []) 2026-02-11T23:30:45.969+0000 7fac4d160640 20 osd.0 94 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:45.969+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 38.805458069s delta_ub 1.980965734s) -- 0x3d0a780 con 0x396b400 2026-02-11T23:30:45.969+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 38.805458069s delta_ub 1.980965734s) -- 0x3d0a900 con 0x3752400 2026-02-11T23:30:45.969+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 38.805458069s delta_ub 4.649149895s) -- 0x3d0af00 con 0x396b800 2026-02-11T23:30:45.969+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 38.805458069s delta_ub 4.649149895s) -- 0x3d0b380 con 0x34c1000 2026-02-11T23:30:45.969+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 38.805458069s delta_ub 7.310458183s) -- 0x3d0b500 con 0x3740000 2026-02-11T23:30:45.969+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 38.805458069s delta_ub 7.310458183s) -- 0x3d0bb00 con 0x3740400 2026-02-11T23:30:45.969+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 38.805458069s delta_ub 9.887004852s) -- 0x3d0ac00 con 0x3740800 2026-02-11T23:30:45.969+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 38.805458069s delta_ub 9.887004852s) -- 0x3da2000 con 0x3740c00 2026-02-11T23:30:45.969+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 13 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 36.824714661s delta_ub -1.980742574s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a780 con 0x396b400 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.969+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980904460s,-1.980742574s]) 2026-02-11T23:30:45.969+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 12 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 34.156543732s delta_ub -4.648913383s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b380 con 0x34c1000 2026-02-11T23:30:45.969+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 10 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 31.495206833s delta_ub -7.310250282s) ==== 2033+0+0 (crc 0 0 0) 0x3d0ac00 con 0x3740400 2026-02-11T23:30:45.969+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 10 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 31.495185852s delta_ub -7.310272217s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a780 con 0x3740000 2026-02-11T23:30:45.969+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:45.969+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.969+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.969+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.969+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649092197s,-4.648913383s]) 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.969+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310463905s,-7.310272217s]) 2026-02-11T23:30:45.969+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 13 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 36.824737549s delta_ub -1.980719447s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b380 con 0x3752400 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.969+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 10 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 28.918706894s delta_ub -9.886751175s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a780 con 0x3740c00 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.969+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310451031s,-7.310250282s]) 2026-02-11T23:30:45.969+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:45.969+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 10 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 28.918680191s delta_ub -9.886775970s) ==== 2033+0+0 (crc 0 0 0) 0x3d0ac00 con 0x3740800 2026-02-11T23:30:45.969+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.969+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.969+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.969+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980933189s,-1.980719447s]) 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.969+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.969+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886981964s,-9.886751175s]) 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.969+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887017250s,-9.886775970s]) 2026-02-11T23:30:45.969+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 12 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:45.971041+0000/38.805458069s send_stamp 34.156570435s delta_ub -4.648889542s) ==== 2033+0+0 (crc 0 0 0) 0x3d0ac00 con 0x396b800 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.969+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.969+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649143219s,-4.648889542s]) 2026-02-11T23:30:45.979+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 9 ==== osd_ping(ping e94 up_from 17 ping_stamp 2026-02-11T23:30:45.980835+0000/31.504871368s send_stamp 31.504871368s delta_ub -7.310250282s) ==== 2033+0+0 (crc 0 0 0) 0x3d0ac00 con 0x3f56c00 2026-02-11T23:30:45.979+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310429573s,-7.310250282s]) 2026-02-11T23:30:45.979+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:45.980835+0000/31.504871368s send_stamp 38.815299988s delta_ub 7.310429573s) -- 0x3d0bb00 con 0x3f56c00 2026-02-11T23:30:45.979+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 9 ==== osd_ping(ping e94 up_from 17 ping_stamp 2026-02-11T23:30:45.980835+0000/31.504871368s send_stamp 31.504871368s delta_ub -7.310250282s) ==== 2033+0+0 (crc 0 0 0) 0x3d0b380 con 0x3f56800 2026-02-11T23:30:45.979+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:45.979+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.979+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.979+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:45.979+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310441494s,-7.310250282s]) 2026-02-11T23:30:45.979+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:45.980835+0000/31.504871368s send_stamp 38.815315247s delta_ub 7.310441494s) -- 0x3da2000 con 0x3f56800 2026-02-11T23:30:45.979+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:45.979+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:45.979+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:45.979+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:46.140+0000 7fac68beb640 10 osd.0 94 tick 2026-02-11T23:30:46.140+0000 7fac68beb640 20 osd.0 94 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:46.167+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:46.167+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:46.167+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:16.169403+0000) 2026-02-11T23:30:46.237+0000 7fac683ea640 10 osd.0 94 tick_without_osd_lock 2026-02-11T23:30:46.237+0000 7fac683ea640 0 osd.0 94 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:46.237+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:46.237+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:46.237+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:46.237+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.113 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:46.237+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:46.239, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:46.237+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:46.237+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:46.237+0000 7fac683ea640 20 osd.0 94 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:46.237+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:46.237+0000 7fac683ea640 20 osd.0 94 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:46.417+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 8 ==== osd_ping(ping e94 up_from 13 ping_stamp 2026-02-11T23:30:46.418614+0000/34.604011536s send_stamp 34.604011536s delta_ub -4.648889542s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a780 con 0x3f56400 2026-02-11T23:30:46.417+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649108410s,-4.648889542s]) 2026-02-11T23:30:46.417+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 8 ==== osd_ping(ping e94 up_from 13 ping_stamp 2026-02-11T23:30:46.418614+0000/34.604011536s send_stamp 34.604011536s delta_ub -4.648889542s) ==== 2033+0+0 (crc 0 0 0) 0x3d0bb00 con 0x3f56000 2026-02-11T23:30:46.417+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:46.418614+0000/34.604011536s send_stamp 39.253120422s delta_ub 4.649108410s) -- 0x3b1b080 con 0x3f56400 2026-02-11T23:30:46.417+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:46.417+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:46.417+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:46.417+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:46.417+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649137974s,-4.648889542s]) 2026-02-11T23:30:46.417+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e94 up_from 5 ping_stamp 2026-02-11T23:30:46.418614+0000/34.604011536s send_stamp 39.253147125s delta_ub 4.649137974s) -- 0x3d0ac00 con 0x3f56000 2026-02-11T23:30:46.417+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:46.417+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:46.417+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:46.417+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:46.469+0000 7fac4d160640 5 osd.0 94 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x6ccce/0xf0000, compress 0x0/0x0/0x0, omap 0x9236, meta 0x1a46dca), peers [1,2,3,4] op hist []) 2026-02-11T23:30:46.469+0000 7fac4d160640 20 osd.0 94 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:46.469+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 39.305629730s delta_ub 1.980933189s) -- 0x3da2180 con 0x396b400 2026-02-11T23:30:46.469+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 39.305629730s delta_ub 1.980933189s) -- 0x3da2300 con 0x3752400 2026-02-11T23:30:46.469+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 39.305629730s delta_ub 4.649137974s) -- 0x3da2480 con 0x396b800 2026-02-11T23:30:46.469+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 39.305629730s delta_ub 4.649137974s) -- 0x3da2600 con 0x34c1000 2026-02-11T23:30:46.469+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 39.305629730s delta_ub 7.310441494s) -- 0x3da2780 con 0x3740000 2026-02-11T23:30:46.469+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 39.305629730s delta_ub 7.310441494s) -- 0x3da2900 con 0x3740400 2026-02-11T23:30:46.469+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 39.305629730s delta_ub 9.887017250s) -- 0x3da2a80 con 0x3740800 2026-02-11T23:30:46.469+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e94 up_from 5 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 39.305629730s delta_ub 9.887017250s) -- 0x3da2c00 con 0x3740c00 2026-02-11T23:30:46.469+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 14 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 37.324836731s delta_ub -1.980793357s) ==== 2033+0+0 (crc 0 0 0) 0x3da2c00 con 0x396b400 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:46.469+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980898857s,-1.980793357s]) 2026-02-11T23:30:46.469+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 11 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 31.995326996s delta_ub -7.310302734s) ==== 2033+0+0 (crc 0 0 0) 0x3da2c00 con 0x3740000 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:46.469+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 14 ==== osd_ping(ping_reply e94 up_from 9 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 37.324851990s delta_ub -1.980774283s) ==== 2033+0+0 (crc 0 0 0) 0x3da2600 con 0x3752400 2026-02-11T23:30:46.469+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310426235s,-7.310302734s]) 2026-02-11T23:30:46.469+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 13 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 34.656681061s delta_ub -4.648944855s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b680 con 0x396b800 2026-02-11T23:30:46.469+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.1 from_epoch 94 2026-02-11T23:30:46.469+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:46.469+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:46.469+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:46.469+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980903149s,-1.980774283s]) 2026-02-11T23:30:46.469+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:46.469+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:46.469+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:46.469+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 13 ==== osd_ping(ping_reply e94 up_from 13 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 34.656711578s delta_ub -4.648919582s) ==== 2033+0+0 (crc 0 0 0) 0x3da2600 con 0x34c1000 2026-02-11T23:30:46.469+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:46.469+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 11 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 29.418834686s delta_ub -9.886795044s) ==== 2033+0+0 (crc 0 0 0) 0x3da2c00 con 0x3740c00 2026-02-11T23:30:46.469+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649099827s,-4.648944855s]) 2026-02-11T23:30:46.469+0000 7fac71bfd640 20 osd.0 94 get_con_osd_cluster to osd.2 from_epoch 94 2026-02-11T23:30:46.469+0000 7fac71bfd640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:46.469+0000 7fac71bfd640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:46.469+0000 7fac71bfd640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:46.469+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 11 ==== osd_ping(ping_reply e94 up_from 17 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 31.995346069s delta_ub -7.310281277s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b680 con 0x3740400 2026-02-11T23:30:46.469+0000 7fac71bfd640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649088383s,-4.648919582s]) 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:46.469+0000 7fac703fa640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:46.469+0000 7fac703fa640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886977196s,-9.886795044s]) 2026-02-11T23:30:46.470+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.3 from_epoch 94 2026-02-11T23:30:46.470+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:46.470+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:46.470+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:46.470+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310479164s,-7.310281277s]) 2026-02-11T23:30:46.470+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 11 ==== osd_ping(ping_reply e94 up_from 21 ping_stamp 2026-02-11T23:30:46.471213+0000/39.305629730s send_stamp 29.418865204s delta_ub -9.886765480s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b680 con 0x3740800 2026-02-11T23:30:46.470+0000 7fac723fe640 20 osd.0 94 get_con_osd_cluster to osd.4 from_epoch 94 2026-02-11T23:30:46.470+0000 7fac723fe640 20 osd.0 94 get_nextmap_reserved map_reservations: {94=1} 2026-02-11T23:30:46.470+0000 7fac723fe640 20 osd.0 94 release_map epoch: 94 2026-02-11T23:30:46.470+0000 7fac723fe640 10 osd.0 94 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 94 is not newer than session's projected_epoch of 94 2026-02-11T23:30:46.470+0000 7fac723fe640 20 osd.0 94 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886983871s,-9.886765480s]) 2026-02-11T23:30:47.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:47.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:47.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:17.169475+0000) 2026-02-11T23:30:47.184+0000 7fac68beb640 10 osd.0 94 tick 2026-02-11T23:30:47.184+0000 7fac68beb640 20 osd.0 94 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:47.252+0000 7fac683ea640 10 osd.0 94 tick_without_osd_lock 2026-02-11T23:30:47.253+0000 7fac683ea640 0 osd.0 94 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:47.253+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:47.253+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:47.253+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:47.253+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.113 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:47.253+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:47.254, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:47.253+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:47.253+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:47.253+0000 7fac683ea640 20 osd.0 94 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:47.253+0000 7fac683ea640 10 osd.0 94 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:47.253+0000 7fac683ea640 20 osd.0 94 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:47.327+0000 7fac6dbf5640 10 monclient: _renew_subs 2026-02-11T23:30:47.327+0000 7fac6dbf5640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:47.327+0000 7fac6dbf5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=95}) -- 0x38f49c0 con 0x396bc00 2026-02-11T23:30:47.852+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 50 ==== osd_map(95..95 src has 1..95) ==== 296+0+0 (crc 0 0 0) 0x38e21c0 con 0x396bc00 2026-02-11T23:30:47.852+0000 7fac626e8640 20 osd.0 94 OSD::ms_dispatch: osd_map(95..95 src has 1..95) 2026-02-11T23:30:47.852+0000 7fac626e8640 20 osd.0 94 _dispatch 0x38e21c0 osd_map(95..95 src has 1..95) 2026-02-11T23:30:47.852+0000 7fac626e8640 3 osd.0 94 handle_osd_map epochs [95,95], i have 94, src has [1,95] 2026-02-11T23:30:47.852+0000 7fac626e8640 10 osd.0 94 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:47.852+0000 7fac626e8640 10 osd.0 94 handle_osd_map got inc map for epoch 95 2026-02-11T23:30:47.852+0000 7fac626e8640 20 osd.0 94 got_full_map 95, nothing requested 2026-02-11T23:30:47.852+0000 7fac626e8640 10 osd.0 94 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 2 2026-02-11T23:30:47.852+0000 7fac626e8640 20 osd.0 94 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:47.852+0000 7fac626e8640 20 osd.0 94 handle_osd_map pg_num_history pg_num_history(e95 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2) 2026-02-11T23:30:47.852+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {95={}} 2026-02-11T23:30:47.852+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:47.852+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e95 maps [1~95] lci=[5,95] tlb=1) 2026-02-11T23:30:47.853+0000 7fac5e983640 10 osd.0 94 _committed_osd_maps 95..95 2026-02-11T23:30:47.853+0000 7fac5e983640 10 osd.0 94 advance to epoch 95 (<= last 95 <= newest_map 95) 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0 95 consume_map version 95 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0 94 await_reserved_maps epoch:95 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0 94 await_reserved_maps done 0 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0 95 consume_map 95 -- publish done 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 3 slots 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:10.0slot:0x2581600 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 10.0 has no pg and waiting_for_split 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0 95 identify_splits_and_merges 4.0 e94 to e95 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0 95 identify_splits_and_merges 4.0 e94 to e95 pg_nums {45=1} 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0 95 identify_splits_and_merges 1.0 e94 to e95 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0 95 identify_splits_and_merges 1.0 e94 to e95 pg_nums {24=1} 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0 95 prune_pg_created 2026-02-11T23:30:47.853+0000 7fac5e983640 10 osd.0:0.consume_map 95 (was 94) 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 10.0 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 10.0 empty, pruning 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:47.853+0000 7fac5e983640 10 osd.0:1.consume_map 95 (was 94) 2026-02-11T23:30:47.853+0000 7fac5e983640 10 osd.0:2.consume_map 95 (was 94) 2026-02-11T23:30:47.853+0000 7fac5e983640 10 osd.0:3.consume_map 95 (was 94) 2026-02-11T23:30:47.853+0000 7fac5e983640 10 osd.0:4.consume_map 95 (was 94) 2026-02-11T23:30:47.853+0000 7fac5e983640 10 osd.0:5.consume_map 95 (was 94) 2026-02-11T23:30:47.853+0000 7fac5e983640 10 osd.0:6.consume_map 95 (was 94) 2026-02-11T23:30:47.853+0000 7fac5e983640 10 osd.0:7.consume_map 95 (was 94) 2026-02-11T23:30:47.853+0000 7fac5e983640 10 osd.0 95 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:47.853+0000 7fac5e983640 15 osd.0 95 enqueue_peering_evt 4.0 epoch_sent: 95 epoch_requested: 95 NullEvt 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2 prio 255 cost 10 e95) 2026-02-11T23:30:47.853+0000 7fac5e983640 15 osd.0 95 enqueue_peering_evt 1.0 epoch_sent: 95 epoch_requested: 95 NullEvt 2026-02-11T23:30:47.853+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2 prio 255 cost 10 e95) 2026-02-11T23:30:47.853+0000 7fac5e983640 7 osd.0 95 activate_map version 95 2026-02-11T23:30:47.853+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:47.853+0000 7fac55971640 20 osd.0 95 agent_entry empty queue 2026-02-11T23:30:47.853+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2immediate prio 255 cost 10 e95) queued 2026-02-11T23:30:47.853+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:47.853+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2immediate prio 255 cost 10 e95) pg 0x4856000 2026-02-11T23:30:47.853+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:47.853+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2immediate prio 255 cost 10 e95) queued 2026-02-11T23:30:47.853+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:47.853+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 95 epoch_requested: 95 NullEvt) class_id 2immediate prio 255 cost 10 e95) pg 0x40f8000 2026-02-11T23:30:47.853+0000 7fac51168640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 95 2026-02-11T23:30:47.853+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 94 -> 95 2026-02-11T23:30:47.853+0000 7fac51168640 10 osd.0 pg_epoch: 94 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:47.853+0000 7fac51168640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:47.853+0000 7fac51168640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:47.853+0000 7fac55170640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 95 2026-02-11T23:30:47.853+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 94 -> 95 2026-02-11T23:30:47.853+0000 7fac55170640 10 osd.0 pg_epoch: 94 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:47.853+0000 7fac51168640 20 osd.0 95 advance_pg new pool opts old pool opts 2026-02-11T23:30:47.853+0000 7fac51168640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 95..95 2026-02-11T23:30:47.853+0000 7fac51168640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:47.853+0000 7fac51168640 20 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 92 while current is 95 2026-02-11T23:30:47.853+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:47.853+0000 7fac55170640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:47.853+0000 7fac51168640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 95 epoch_requested: 95 NullEvt 2026-02-11T23:30:47.853+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:47.853+0000 7fac55170640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:47.853+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:47.853+0000 7fac55170640 20 osd.0 95 advance_pg new pool opts old pool opts 2026-02-11T23:30:47.853+0000 7fac55170640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 95..95 2026-02-11T23:30:47.853+0000 7fac55170640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:47.853+0000 7fac55170640 20 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 95 2026-02-11T23:30:47.853+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:47.853+0000 7fac55170640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 95 epoch_requested: 95 NullEvt 2026-02-11T23:30:47.853+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:47.853+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.011+0000 7fac60ee5640 20 osd.0 95 reports for 0 queries 2026-02-11T23:30:48.011+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x34c3180 con 0x396a800 2026-02-11T23:30:48.011+0000 7fac60ee5640 15 osd.0 95 collect_pg_stats 2026-02-11T23:30:48.011+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836491 v 0) -- 0x3d9e900 con 0x396a800 2026-02-11T23:30:48.134+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 51 ==== osd_map(95..96 src has 1..96) ==== 556+0+0 (crc 0 0 0) 0x46ea540 con 0x396bc00 2026-02-11T23:30:48.134+0000 7fac626e8640 20 osd.0 95 OSD::ms_dispatch: osd_map(95..96 src has 1..96) 2026-02-11T23:30:48.134+0000 7fac626e8640 20 osd.0 95 _dispatch 0x46ea540 osd_map(95..96 src has 1..96) 2026-02-11T23:30:48.134+0000 7fac626e8640 3 osd.0 95 handle_osd_map epochs [95,96], i have 95, src has [1,96] 2026-02-11T23:30:48.134+0000 7fac626e8640 10 osd.0 95 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:48.134+0000 7fac626e8640 10 osd.0 95 handle_osd_map got inc map for epoch 96 2026-02-11T23:30:48.134+0000 7fac626e8640 20 osd.0 95 got_full_map 96, nothing requested 2026-02-11T23:30:48.134+0000 7fac626e8640 10 osd.0 95 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 3 2026-02-11T23:30:48.134+0000 7fac626e8640 20 osd.0 95 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:48.134+0000 7fac626e8640 20 osd.0 95 handle_osd_map pg_num_history pg_num_history(e96 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3) 2026-02-11T23:30:48.134+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {96={}} 2026-02-11T23:30:48.134+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:48.134+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e96 maps [1~96] lci=[5,96] tlb=1) 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0 95 _committed_osd_maps 96..96 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0 95 advance to epoch 96 (<= last 96 <= newest_map 96) 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0 96 consume_map version 96 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0 95 await_reserved_maps epoch:96 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0 95 await_reserved_maps done 0 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0 96 consume_map 96 -- publish done 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0 96 identify_splits_and_merges 4.0 e95 to e96 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0 96 identify_splits_and_merges 4.0 e95 to e96 pg_nums {45=1} 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0 96 identify_splits_and_merges 1.0 e95 to e96 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0 96 identify_splits_and_merges 1.0 e95 to e96 pg_nums {24=1} 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0 96 prune_pg_created 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0:0.consume_map 96 (was 95) 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0:1.consume_map 96 (was 95) 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0:2.consume_map 96 (was 95) 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0:3.consume_map 96 (was 95) 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0:4.consume_map 96 (was 95) 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0:5.consume_map 96 (was 95) 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0:6.consume_map 96 (was 95) 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0:7.consume_map 96 (was 95) 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0 96 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:48.134+0000 7fac5e983640 15 osd.0 96 enqueue_peering_evt 4.0 epoch_sent: 96 epoch_requested: 96 NullEvt 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2 prio 255 cost 10 e96) 2026-02-11T23:30:48.134+0000 7fac5e983640 15 osd.0 96 enqueue_peering_evt 1.0 epoch_sent: 96 epoch_requested: 96 NullEvt 2026-02-11T23:30:48.134+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2 prio 255 cost 10 e96) 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0 96 maybe_update_heartbeat_peers forcing update after 20.077364 seconds 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0 96 reset_heartbeat_peers 2026-02-11T23:30:48.134+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:48.134+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2immediate prio 255 cost 10 e96) queued 2026-02-11T23:30:48.134+0000 7fac5e983640 10 osd.0 96 maybe_update_heartbeat_peers updating 2026-02-11T23:30:48.134+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:48.135+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2immediate prio 255 cost 10 e96) pg 0x4856000 2026-02-11T23:30:48.135+0000 7fac5e983640 10 osd.0 96 adding neighbor peer osd.1 2026-02-11T23:30:48.135+0000 7fac5e983640 10 osd.0 96 adding neighbor peer osd.4 2026-02-11T23:30:48.135+0000 7fac5e983640 10 osd.0 96 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:30:48.135+0000 7fac5e983640 7 osd.0 96 activate_map version 96 2026-02-11T23:30:48.135+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:48.135+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2immediate prio 255 cost 10 e96) queued 2026-02-11T23:30:48.135+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:48.135+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 96 epoch_requested: 96 NullEvt) class_id 2immediate prio 255 cost 10 e96) pg 0x40f8000 2026-02-11T23:30:48.135+0000 7fac51168640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 96 2026-02-11T23:30:48.135+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 95 -> 96 2026-02-11T23:30:48.135+0000 7fac51168640 10 osd.0 pg_epoch: 95 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:48.135+0000 7fac55971640 20 osd.0 96 agent_entry empty queue 2026-02-11T23:30:48.135+0000 7fac55170640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 96 2026-02-11T23:30:48.135+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 95 -> 96 2026-02-11T23:30:48.135+0000 7fac55170640 10 osd.0 pg_epoch: 95 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:48.135+0000 7fac51168640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:48.135+0000 7fac51168640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:48.135+0000 7fac55170640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:48.135+0000 7fac55170640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:48.135+0000 7fac51168640 20 osd.0 96 advance_pg new pool opts old pool opts 2026-02-11T23:30:48.135+0000 7fac55170640 20 osd.0 96 advance_pg new pool opts old pool opts 2026-02-11T23:30:48.135+0000 7fac55170640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 96..96 2026-02-11T23:30:48.135+0000 7fac55170640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:48.135+0000 7fac51168640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 96..96 2026-02-11T23:30:48.135+0000 7fac55170640 20 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 96 2026-02-11T23:30:48.135+0000 7fac51168640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:48.135+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:48.135+0000 7fac51168640 20 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 92 while current is 96 2026-02-11T23:30:48.135+0000 7fac55170640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 96 epoch_requested: 96 NullEvt 2026-02-11T23:30:48.135+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:48.135+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:48.135+0000 7fac51168640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 96 epoch_requested: 96 NullEvt 2026-02-11T23:30:48.135+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:48.135+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.135+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:48.161+0000 7fac68beb640 10 osd.0 96 tick 2026-02-11T23:30:48.161+0000 7fac68beb640 20 osd.0 96 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:48.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:48.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:48.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:18.169530+0000) 2026-02-11T23:30:48.246+0000 7fac683ea640 10 osd.0 96 tick_without_osd_lock 2026-02-11T23:30:48.247+0000 7fac683ea640 0 osd.0 96 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:48.247+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:48.247+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:48.247+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:48.247+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.113 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:48.247+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:48.248, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:48.247+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:48.247+0000 7fac683ea640 10 osd.0 96 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:48.247+0000 7fac683ea640 20 osd.0 96 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:48.247+0000 7fac683ea640 10 osd.0 96 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:48.247+0000 7fac683ea640 20 osd.0 96 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:49.154+0000 7fac68beb640 10 osd.0 96 tick 2026-02-11T23:30:49.154+0000 7fac68beb640 20 osd.0 96 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:49.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:49.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:49.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:19.169552+0000) 2026-02-11T23:30:49.267+0000 7fac683ea640 10 osd.0 96 tick_without_osd_lock 2026-02-11T23:30:49.267+0000 7fac683ea640 0 osd.0 96 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:49.267+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:49.267+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.113 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:49.267+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:49.268, recovery is active?:false restrictions:<....> 2026-02-11T23:30:49.267+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:49.267+0000 7fac683ea640 10 osd.0 96 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:49.267+0000 7fac683ea640 20 osd.0 96 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:49.267+0000 7fac683ea640 10 osd.0 96 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:49.267+0000 7fac683ea640 20 osd.0 96 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:49.556+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 11 ==== osd_ping(ping e99 up_from 21 ping_stamp 2026-02-11T23:30:49.557668+0000/32.505172729s send_stamp 32.505172729s delta_ub -9.886765480s) ==== 2033+0+0 (crc 0 0 0) 0x3da2c00 con 0x3f57400 2026-02-11T23:30:49.556+0000 7fac703fa640 20 osd.0 96 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887003899s,-9.886765480s]) 2026-02-11T23:30:49.556+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 11 ==== osd_ping(ping e99 up_from 21 ping_stamp 2026-02-11T23:30:49.557668+0000/32.505172729s send_stamp 32.505172729s delta_ub -9.886765480s) ==== 2033+0+0 (crc 0 0 0) 0x3da2600 con 0x3f57800 2026-02-11T23:30:49.556+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e96 up_from 5 ping_stamp 2026-02-11T23:30:49.557668+0000/32.505172729s send_stamp 42.392177582s delta_ub 9.887003899s) -- 0x3da2480 con 0x3f57400 2026-02-11T23:30:49.556+0000 7fac703fa640 20 osd.0 96 get_con_osd_cluster to osd.4 from_epoch 96 2026-02-11T23:30:49.556+0000 7fac703fa640 20 osd.0 96 get_nextmap_reserved map_reservations: {96=1} 2026-02-11T23:30:49.556+0000 7fac703fa640 20 osd.0 96 release_map epoch: 96 2026-02-11T23:30:49.556+0000 7fac703fa640 10 osd.0 96 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 94 to ping map epoch of 99 2026-02-11T23:30:49.556+0000 7fac703fa640 10 osd.0 96 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 96 is not newer than session's projected_epoch of 99 2026-02-11T23:30:49.556+0000 7fac71bfd640 20 osd.0 96 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887034416s,-9.886765480s]) 2026-02-11T23:30:49.556+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e96 up_from 5 ping_stamp 2026-02-11T23:30:49.557668+0000/32.505172729s send_stamp 42.392208099s delta_ub 9.887034416s) -- 0x3da3980 con 0x3f57800 2026-02-11T23:30:49.556+0000 7fac71bfd640 20 osd.0 96 get_con_osd_cluster to osd.4 from_epoch 96 2026-02-11T23:30:49.556+0000 7fac71bfd640 20 osd.0 96 get_nextmap_reserved map_reservations: {96=1} 2026-02-11T23:30:49.556+0000 7fac71bfd640 20 osd.0 96 release_map epoch: 96 2026-02-11T23:30:49.556+0000 7fac71bfd640 10 osd.0 96 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 96 is not newer than session's projected_epoch of 99 2026-02-11T23:30:49.556+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 31 ==== osd_map(97..99 src has 1..99) ==== 816+0+0 (unknown 0 0 0) 0x3941500 con 0x3e4a000 2026-02-11T23:30:49.556+0000 7fac606e4640 20 osd.0 96 OSD::ms_dispatch: osd_map(97..99 src has 1..99) 2026-02-11T23:30:49.556+0000 7fac606e4640 20 osd.0 96 _dispatch 0x3941500 osd_map(97..99 src has 1..99) 2026-02-11T23:30:49.556+0000 7fac606e4640 3 osd.0 96 handle_osd_map epochs [97,99], i have 96, src has [1,99] 2026-02-11T23:30:49.556+0000 7fac606e4640 10 osd.0 96 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:49.556+0000 7fac606e4640 10 osd.0 96 handle_osd_map got inc map for epoch 97 2026-02-11T23:30:49.556+0000 7fac606e4640 20 osd.0 96 got_full_map 97, nothing requested 2026-02-11T23:30:49.556+0000 7fac606e4640 10 osd.0 96 handle_osd_map got inc map for epoch 98 2026-02-11T23:30:49.556+0000 7fac606e4640 20 osd.0 96 got_full_map 98, nothing requested 2026-02-11T23:30:49.556+0000 7fac606e4640 10 osd.0 96 handle_osd_map got inc map for epoch 99 2026-02-11T23:30:49.557+0000 7fac606e4640 20 osd.0 96 got_full_map 99, nothing requested 2026-02-11T23:30:49.557+0000 7fac606e4640 10 osd.0 96 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 5 2026-02-11T23:30:49.557+0000 7fac606e4640 10 osd.0 96 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 6 2026-02-11T23:30:49.557+0000 7fac606e4640 10 osd.0 96 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 7 2026-02-11T23:30:49.557+0000 7fac606e4640 20 osd.0 96 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:49.557+0000 7fac606e4640 20 osd.0 96 handle_osd_map pg_num_history pg_num_history(e99 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7) 2026-02-11T23:30:49.557+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {97={},98={},99={}} 2026-02-11T23:30:49.557+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:49.557+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e99 maps [1~99] lci=[5,99] tlb=1) 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0 96 _committed_osd_maps 97..99 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0 96 advance to epoch 97 (<= last 99 <= newest_map 99) 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0 97 advance to epoch 98 (<= last 99 <= newest_map 99) 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0 98 advance to epoch 99 (<= last 99 <= newest_map 99) 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0 99 consume_map version 99 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0 96 await_reserved_maps epoch:99 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0 96 await_reserved_maps done 0 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0 99 consume_map 99 -- publish done 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0 99 identify_splits_and_merges 4.0 e96 to e99 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0 99 identify_splits_and_merges 4.0 e96 to e99 pg_nums {45=1} 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0 99 identify_splits_and_merges 1.0 e96 to e99 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0 99 identify_splits_and_merges 1.0 e96 to e99 pg_nums {24=1} 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0 99 prune_pg_created 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0:0.consume_map 99 (was 96) 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0:1.consume_map 99 (was 96) 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0:2.consume_map 99 (was 96) 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0:3.consume_map 99 (was 96) 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0:4.consume_map 99 (was 96) 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0:5.consume_map 99 (was 96) 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0:6.consume_map 99 (was 96) 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0:7.consume_map 99 (was 96) 2026-02-11T23:30:49.557+0000 7fac5e983640 10 osd.0 99 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:49.557+0000 7fac5e983640 15 osd.0 99 enqueue_peering_evt 4.0 epoch_sent: 99 epoch_requested: 99 NullEvt 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2 prio 255 cost 10 e99) 2026-02-11T23:30:49.557+0000 7fac5e983640 15 osd.0 99 enqueue_peering_evt 1.0 epoch_sent: 99 epoch_requested: 99 NullEvt 2026-02-11T23:30:49.557+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2 prio 255 cost 10 e99) 2026-02-11T23:30:49.557+0000 7fac5e983640 7 osd.0 99 activate_map version 99 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2immediate prio 255 cost 10 e99) queued 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2immediate prio 255 cost 10 e99) pg 0x4856000 2026-02-11T23:30:49.557+0000 7fac55971640 20 osd.0 99 agent_entry empty queue 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2immediate prio 255 cost 10 e99) queued 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 97 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 99 epoch_requested: 99 NullEvt) class_id 2immediate prio 255 cost 10 e99) pg 0x40f8000 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 96 -> 97 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 96 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 97 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 97 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 96 -> 97 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 97 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 96 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0 99 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 97 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 97 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 98 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 97 -> 98 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 97 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 97 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0 99 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 99 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0 99 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 98 -> 99 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 98 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 97 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 98 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 97 -> 98 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 97 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0 99 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 97..99 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0 99 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 92 while current is 99 2026-02-11T23:30:49.557+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 99 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 98 -> 99 2026-02-11T23:30:49.557+0000 7fac51168640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 99 epoch_requested: 99 NullEvt 2026-02-11T23:30:49.557+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 98 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:49.557+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0 99 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 97..99 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 99 2026-02-11T23:30:49.557+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:49.557+0000 7fac55170640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 99 epoch_requested: 99 NullEvt 2026-02-11T23:30:49.557+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:49.557+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:49.849+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 13 ==== osd_ping(ping e102 up_from 9 ping_stamp 2026-02-11T23:30:49.851131+0000/40.704696655s send_stamp 40.704696655s delta_ub -1.980774283s) ==== 2033+0+0 (crc 0 0 0) 0x3da3980 con 0x396a400 2026-02-11T23:30:49.849+0000 7fac71bfd640 20 osd.0 99 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980934024s,-1.980774283s]) 2026-02-11T23:30:49.849+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e99 up_from 5 ping_stamp 2026-02-11T23:30:49.851131+0000/40.704696655s send_stamp 42.685626984s delta_ub 1.980934024s) -- 0x3da2600 con 0x396a400 2026-02-11T23:30:49.849+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 13 ==== osd_ping(ping e102 up_from 9 ping_stamp 2026-02-11T23:30:49.851131+0000/40.704696655s send_stamp 40.704696655s delta_ub -1.980774283s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a480 con 0x396a000 2026-02-11T23:30:49.849+0000 7fac71bfd640 20 osd.0 99 get_con_osd_cluster to osd.1 from_epoch 99 2026-02-11T23:30:49.849+0000 7fac71bfd640 20 osd.0 99 get_nextmap_reserved map_reservations: {99=1} 2026-02-11T23:30:49.849+0000 7fac71bfd640 20 osd.0 99 release_map epoch: 99 2026-02-11T23:30:49.849+0000 7fac71bfd640 10 osd.0 99 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 94 to ping map epoch of 102 2026-02-11T23:30:49.849+0000 7fac71bfd640 10 osd.0 99 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 99 is not newer than session's projected_epoch of 102 2026-02-11T23:30:49.849+0000 7fac723fe640 20 osd.0 99 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980958104s,-1.980774283s]) 2026-02-11T23:30:49.849+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e99 up_from 5 ping_stamp 2026-02-11T23:30:49.851131+0000/40.704696655s send_stamp 42.685653687s delta_ub 1.980958104s) -- 0x3b1b800 con 0x396a000 2026-02-11T23:30:49.849+0000 7fac723fe640 20 osd.0 99 get_con_osd_cluster to osd.1 from_epoch 99 2026-02-11T23:30:49.849+0000 7fac723fe640 20 osd.0 99 get_nextmap_reserved map_reservations: {99=1} 2026-02-11T23:30:49.849+0000 7fac723fe640 20 osd.0 99 release_map epoch: 99 2026-02-11T23:30:49.849+0000 7fac723fe640 10 osd.0 99 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 99 is not newer than session's projected_epoch of 102 2026-02-11T23:30:49.850+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 24 ==== osd_map(100..102 src has 1..102) ==== 816+0+0 (unknown 0 0 0) 0x3941180 con 0x3752000 2026-02-11T23:30:49.850+0000 7fac606e4640 20 osd.0 99 OSD::ms_dispatch: osd_map(100..102 src has 1..102) 2026-02-11T23:30:49.850+0000 7fac606e4640 20 osd.0 99 _dispatch 0x3941180 osd_map(100..102 src has 1..102) 2026-02-11T23:30:49.850+0000 7fac606e4640 3 osd.0 99 handle_osd_map epochs [100,102], i have 99, src has [1,102] 2026-02-11T23:30:49.850+0000 7fac606e4640 10 osd.0 99 handle_osd_map: received 0 full maps and 3 incremental maps 2026-02-11T23:30:49.850+0000 7fac606e4640 10 osd.0 99 handle_osd_map got inc map for epoch 100 2026-02-11T23:30:49.850+0000 7fac606e4640 20 osd.0 99 got_full_map 100, nothing requested 2026-02-11T23:30:49.850+0000 7fac606e4640 10 osd.0 99 handle_osd_map got inc map for epoch 101 2026-02-11T23:30:49.850+0000 7fac606e4640 20 osd.0 99 got_full_map 101, nothing requested 2026-02-11T23:30:49.850+0000 7fac606e4640 10 osd.0 99 handle_osd_map got inc map for epoch 102 2026-02-11T23:30:49.850+0000 7fac606e4640 20 osd.0 99 got_full_map 102, nothing requested 2026-02-11T23:30:49.850+0000 7fac606e4640 10 osd.0 99 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 8 2026-02-11T23:30:49.850+0000 7fac606e4640 10 osd.0 99 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 9 2026-02-11T23:30:49.850+0000 7fac606e4640 10 osd.0 99 _track_pools_and_pg_num_changes recording final pg_pool_t for pool 10 2026-02-11T23:30:49.850+0000 7fac606e4640 20 osd.0 99 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:49.850+0000 7fac606e4640 20 osd.0 99 handle_osd_map pg_num_history pg_num_history(e102 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:30:49.850+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {100={},101={},102={}} 2026-02-11T23:30:49.850+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:49.850+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e102 maps [1~102] lci=[5,102] tlb=1) 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0 99 _committed_osd_maps 100..102 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0 99 advance to epoch 100 (<= last 102 <= newest_map 102) 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0 100 advance to epoch 101 (<= last 102 <= newest_map 102) 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0 101 advance to epoch 102 (<= last 102 <= newest_map 102) 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0 102 consume_map version 102 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0 99 await_reserved_maps epoch:102 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0 99 await_reserved_maps done 0 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0 102 consume_map 102 -- publish done 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0 102 identify_splits_and_merges 4.0 e99 to e102 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0 102 identify_splits_and_merges 4.0 e99 to e102 pg_nums {45=1} 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0 102 identify_splits_and_merges 1.0 e99 to e102 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0 102 identify_splits_and_merges 1.0 e99 to e102 pg_nums {24=1} 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0 102 prune_pg_created 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0:0.consume_map 102 (was 99) 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0:1.consume_map 102 (was 99) 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0:2.consume_map 102 (was 99) 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0:3.consume_map 102 (was 99) 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0:4.consume_map 102 (was 99) 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0:5.consume_map 102 (was 99) 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0:6.consume_map 102 (was 99) 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0:7.consume_map 102 (was 99) 2026-02-11T23:30:49.850+0000 7fac5e983640 10 osd.0 102 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:49.850+0000 7fac5e983640 15 osd.0 102 enqueue_peering_evt 4.0 epoch_sent: 102 epoch_requested: 102 NullEvt 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 102 epoch_requested: 102 NullEvt) class_id 2 prio 255 cost 10 e102) 2026-02-11T23:30:49.850+0000 7fac5e983640 15 osd.0 102 enqueue_peering_evt 1.0 epoch_sent: 102 epoch_requested: 102 NullEvt 2026-02-11T23:30:49.850+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 102 epoch_requested: 102 NullEvt) class_id 2 prio 255 cost 10 e102) 2026-02-11T23:30:49.850+0000 7fac5e983640 7 osd.0 102 activate_map version 102 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 102 epoch_requested: 102 NullEvt) class_id 2immediate prio 255 cost 10 e102) queued 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 102 epoch_requested: 102 NullEvt) class_id 2immediate prio 255 cost 10 e102) pg 0x4856000 2026-02-11T23:30:49.850+0000 7fac55971640 20 osd.0 102 agent_entry empty queue 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 102 epoch_requested: 102 NullEvt) class_id 2immediate prio 255 cost 10 e102) queued 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 102 epoch_requested: 102 NullEvt) class_id 2immediate prio 255 cost 10 e102) pg 0x40f8000 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 100 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 99 -> 100 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 99 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 100 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 100 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 100 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 99 -> 100 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 99 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0 102 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 100 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 101 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 100 -> 101 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 100 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 100 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 101 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 100 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 101 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0 102 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0 102 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 101 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 102 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 101 -> 102 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 101 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 100 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 101 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 100 -> 101 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 100 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0 102 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 100..102 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 101 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 101 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 92 while current is 102 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0 102 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.850+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:49.850+0000 7fac51168640 10 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 102 epoch_requested: 102 NullEvt 2026-02-11T23:30:49.850+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 101 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 102 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 101 -> 102 2026-02-11T23:30:49.850+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 101 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0 102 advance_pg new pool opts old pool opts 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_activate_map: epoch range: 100..102 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 102 2026-02-11T23:30:49.850+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:49.850+0000 7fac55170640 10 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 102 epoch_requested: 102 NullEvt 2026-02-11T23:30:49.850+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:49.850+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:49.969+0000 7fac4d160640 5 osd.0 102 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x6ffd7/0xf8000, compress 0x0/0x0/0x0, omap 0x974c, meta 0x1a468b4), peers [1,2,3,4] op hist []) 2026-02-11T23:30:49.969+0000 7fac4d160640 20 osd.0 102 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:49.969+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e102 up_from 5 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 42.805805206s delta_ub 1.980958104s) -- 0x3da2d80 con 0x396b400 2026-02-11T23:30:49.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e102 up_from 5 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 42.805805206s delta_ub 1.980958104s) -- 0x3da2f00 con 0x3752400 2026-02-11T23:30:49.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e102 up_from 5 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 42.805805206s delta_ub 4.649088383s) -- 0x3da3080 con 0x396b800 2026-02-11T23:30:49.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e102 up_from 5 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 42.805805206s delta_ub 4.649088383s) -- 0x3da3200 con 0x34c1000 2026-02-11T23:30:49.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e102 up_from 5 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 42.805805206s delta_ub 7.310479164s) -- 0x3da3380 con 0x3740000 2026-02-11T23:30:49.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e102 up_from 5 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 42.805805206s delta_ub 7.310479164s) -- 0x3da3500 con 0x3740400 2026-02-11T23:30:49.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e102 up_from 5 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 42.805805206s delta_ub 9.887034416s) -- 0x3da3800 con 0x3740800 2026-02-11T23:30:49.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e102 up_from 5 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 42.805805206s delta_ub 9.887034416s) -- 0x3da3680 con 0x3740c00 2026-02-11T23:30:49.970+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 14 ==== osd_ping(ping_reply e102 up_from 13 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 38.156887054s delta_ub -4.648919582s) ==== 2033+0+0 (crc 0 0 0) 0x3da3080 con 0x396b800 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 get_con_osd_cluster to osd.2 from_epoch 102 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 get_nextmap_reserved map_reservations: {102=1} 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 release_map epoch: 102 2026-02-11T23:30:49.970+0000 7fac723fe640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 94 to ping map epoch of 102 2026-02-11T23:30:49.970+0000 7fac723fe640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 102 is not newer than session's projected_epoch of 102 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649074554s,-4.648919582s]) 2026-02-11T23:30:49.970+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 12 ==== osd_ping(ping_reply e102 up_from 17 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 35.495532990s delta_ub -7.310269356s) ==== 2033+0+0 (crc 0 0 0) 0x3da3680 con 0x3740000 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 get_con_osd_cluster to osd.3 from_epoch 102 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 get_nextmap_reserved map_reservations: {102=1} 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 release_map epoch: 102 2026-02-11T23:30:49.970+0000 7fac703fa640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 94 to ping map epoch of 102 2026-02-11T23:30:49.970+0000 7fac703fa640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 102 is not newer than session's projected_epoch of 102 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310437202s,-7.310269356s]) 2026-02-11T23:30:49.970+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 15 ==== osd_ping(ping_reply e102 up_from 9 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 40.825080872s delta_ub -1.980722070s) ==== 2033+0+0 (crc 0 0 0) 0x3da3200 con 0x3752400 2026-02-11T23:30:49.970+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 15 ==== osd_ping(ping_reply e102 up_from 9 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 40.825061798s delta_ub -1.980746031s) ==== 2033+0+0 (crc 0 0 0) 0x3da3680 con 0x396b400 2026-02-11T23:30:49.970+0000 7fac71bfd640 20 osd.0 102 get_con_osd_cluster to osd.1 from_epoch 102 2026-02-11T23:30:49.970+0000 7fac71bfd640 20 osd.0 102 get_nextmap_reserved map_reservations: {102=1} 2026-02-11T23:30:49.970+0000 7fac71bfd640 20 osd.0 102 release_map epoch: 102 2026-02-11T23:30:49.970+0000 7fac71bfd640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 102 is not newer than session's projected_epoch of 102 2026-02-11T23:30:49.970+0000 7fac71bfd640 20 osd.0 102 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980907321s,-1.980722070s]) 2026-02-11T23:30:49.970+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 12 ==== osd_ping(ping_reply e102 up_from 17 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 35.495574951s delta_ub -7.310232162s) ==== 2033+0+0 (crc 0 0 0) 0x3da3080 con 0x3740400 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 get_con_osd_cluster to osd.3 from_epoch 102 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 get_nextmap_reserved map_reservations: {102=1} 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 release_map epoch: 102 2026-02-11T23:30:49.970+0000 7fac723fe640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 102 is not newer than session's projected_epoch of 102 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310427189s,-7.310232162s]) 2026-02-11T23:30:49.970+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 14 ==== osd_ping(ping_reply e102 up_from 13 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 38.156932831s delta_ub -4.648873329s) ==== 2033+0+0 (crc 0 0 0) 0x3da3200 con 0x34c1000 2026-02-11T23:30:49.970+0000 7fac71bfd640 20 osd.0 102 get_con_osd_cluster to osd.2 from_epoch 102 2026-02-11T23:30:49.970+0000 7fac71bfd640 20 osd.0 102 get_nextmap_reserved map_reservations: {102=1} 2026-02-11T23:30:49.970+0000 7fac71bfd640 20 osd.0 102 release_map epoch: 102 2026-02-11T23:30:49.970+0000 7fac71bfd640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 102 is not newer than session's projected_epoch of 102 2026-02-11T23:30:49.970+0000 7fac71bfd640 20 osd.0 102 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649079323s,-4.648873329s]) 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 get_con_osd_cluster to osd.1 from_epoch 102 2026-02-11T23:30:49.970+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 12 ==== osd_ping(ping_reply e102 up_from 21 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 32.919078827s delta_ub -9.886726379s) ==== 2033+0+0 (crc 0 0 0) 0x3da3080 con 0x3740800 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 get_nextmap_reserved map_reservations: {102=1} 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 release_map epoch: 102 2026-02-11T23:30:49.970+0000 7fac703fa640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 102 is not newer than session's projected_epoch of 102 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980962873s,-1.980746031s]) 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 get_con_osd_cluster to osd.4 from_epoch 102 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 get_nextmap_reserved map_reservations: {102=1} 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 release_map epoch: 102 2026-02-11T23:30:49.970+0000 7fac723fe640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 99 to ping map epoch of 102 2026-02-11T23:30:49.970+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 12 ==== osd_ping(ping_reply e102 up_from 21 ping_stamp 2026-02-11T23:30:49.971390+0000/42.805805206s send_stamp 32.919055939s delta_ub -9.886751175s) ==== 2033+0+0 (crc 0 0 0) 0x3da3680 con 0x3740c00 2026-02-11T23:30:49.970+0000 7fac723fe640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 102 is not newer than session's projected_epoch of 102 2026-02-11T23:30:49.970+0000 7fac723fe640 20 osd.0 102 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886957169s,-9.886726379s]) 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 get_con_osd_cluster to osd.4 from_epoch 102 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 get_nextmap_reserved map_reservations: {102=1} 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 release_map epoch: 102 2026-02-11T23:30:49.970+0000 7fac703fa640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 102 is not newer than session's projected_epoch of 102 2026-02-11T23:30:49.970+0000 7fac703fa640 20 osd.0 102 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886990547s,-9.886751175s]) 2026-02-11T23:30:50.080+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 10 ==== osd_ping(ping e102 up_from 17 ping_stamp 2026-02-11T23:30:50.081027+0000/35.605060577s send_stamp 35.605060577s delta_ub -7.310232162s) ==== 2033+0+0 (crc 0 0 0) 0x3da3080 con 0x3f56c00 2026-02-11T23:30:50.080+0000 7fac723fe640 20 osd.0 102 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310437202s,-7.310232162s]) 2026-02-11T23:30:50.080+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 10 ==== osd_ping(ping e102 up_from 17 ping_stamp 2026-02-11T23:30:50.081027+0000/35.605060577s send_stamp 35.605060577s delta_ub -7.310232162s) ==== 2033+0+0 (crc 0 0 0) 0x3da3200 con 0x3f56800 2026-02-11T23:30:50.080+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e102 up_from 5 ping_stamp 2026-02-11T23:30:50.081027+0000/35.605060577s send_stamp 42.915500641s delta_ub 7.310437202s) -- 0x3da3500 con 0x3f56c00 2026-02-11T23:30:50.080+0000 7fac723fe640 20 osd.0 102 get_con_osd_cluster to osd.3 from_epoch 102 2026-02-11T23:30:50.080+0000 7fac723fe640 20 osd.0 102 get_nextmap_reserved map_reservations: {102=1} 2026-02-11T23:30:50.080+0000 7fac723fe640 20 osd.0 102 release_map epoch: 102 2026-02-11T23:30:50.080+0000 7fac723fe640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 102 is not newer than session's projected_epoch of 102 2026-02-11T23:30:50.080+0000 7fac71bfd640 20 osd.0 102 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310455799s,-7.310232162s]) 2026-02-11T23:30:50.080+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e102 up_from 5 ping_stamp 2026-02-11T23:30:50.081027+0000/35.605060577s send_stamp 42.915519714s delta_ub 7.310455799s) -- 0x3da2f00 con 0x3f56800 2026-02-11T23:30:50.080+0000 7fac71bfd640 20 osd.0 102 get_con_osd_cluster to osd.3 from_epoch 102 2026-02-11T23:30:50.080+0000 7fac71bfd640 20 osd.0 102 get_nextmap_reserved map_reservations: {102=1} 2026-02-11T23:30:50.080+0000 7fac71bfd640 20 osd.0 102 release_map epoch: 102 2026-02-11T23:30:50.080+0000 7fac71bfd640 10 osd.0 102 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 102 is not newer than session's projected_epoch of 102 2026-02-11T23:30:50.105+0000 7fac68beb640 10 osd.0 102 tick 2026-02-11T23:30:50.105+0000 7fac68beb640 20 osd.0 102 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:50.156+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 32 ==== pg_lease(4.0 pg_lease(ru 41.104904175s ub 49.105007172s int 16.000000000s) e103/103) ==== 58+0+0 (unknown 0 0 0) 0x3da3680 con 0x3e4a000 2026-02-11T23:30:50.156+0000 7fac703fa640 15 osd.0 102 enqueue_peering_evt 4.0 epoch_sent: 103 epoch_requested: 103 MLease epoch 103 from osd.4 pg_lease(ru 41.104904175s ub 49.105007172s int 16.000000000s) 2026-02-11T23:30:50.156+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 MLease epoch 103 from osd.4 pg_lease(ru 41.104904175s ub 49.105007172s int 16.000000000s)) class_id 2 prio 255 cost 10 e103) 2026-02-11T23:30:50.156+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.156+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 MLease epoch 103 from osd.4 pg_lease(ru 41.104904175s ub 49.105007172s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e103) queued 2026-02-11T23:30:50.156+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.156+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 MLease epoch 103 from osd.4 pg_lease(ru 41.104904175s ub 49.105007172s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e103) pg 0x4856000 2026-02-11T23:30:50.156+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 103, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 MLease epoch 103 from osd.4 pg_lease(ru 41.104904175s ub 49.105007172s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e103) 2026-02-11T23:30:50.156+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.156+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:50.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:50.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:20.169614+0000) 2026-02-11T23:30:50.168+0000 7fac616e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:30:50.261+0000 7fac683ea640 10 osd.0 102 tick_without_osd_lock 2026-02-11T23:30:50.261+0000 7fac683ea640 0 osd.0 102 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:50.261+0000 7fac683ea640 20 osd.0 102 tick_without_osd_lock max_waiting_epoch 103, requesting new map 2026-02-11T23:30:50.261+0000 7fac683ea640 10 monclient: _renew_subs 2026-02-11T23:30:50.261+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:50.261+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=103}) -- 0x3e56820 con 0x396bc00 2026-02-11T23:30:50.261+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:50.261+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:50.261+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:50.261+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.107 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:50.261+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:50.262, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:50.261+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:50.261+0000 7fac683ea640 10 osd.0 102 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:50.261+0000 7fac683ea640 20 osd.0 102 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:50.261+0000 7fac683ea640 10 osd.0 102 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:50.261+0000 7fac683ea640 20 osd.0 102 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:50.261+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 52 ==== osd_map(103..104 src has 1..104) ==== 1200+0+0 (crc 0 0 0) 0x46ea1c0 con 0x396bc00 2026-02-11T23:30:50.261+0000 7fac626e8640 20 osd.0 102 OSD::ms_dispatch: osd_map(103..104 src has 1..104) 2026-02-11T23:30:50.261+0000 7fac626e8640 20 osd.0 102 _dispatch 0x46ea1c0 osd_map(103..104 src has 1..104) 2026-02-11T23:30:50.261+0000 7fac626e8640 3 osd.0 102 handle_osd_map epochs [103,104], i have 102, src has [1,104] 2026-02-11T23:30:50.261+0000 7fac626e8640 10 osd.0 102 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:50.261+0000 7fac626e8640 10 osd.0 102 handle_osd_map got inc map for epoch 103 2026-02-11T23:30:50.261+0000 7fac626e8640 20 osd.0 102 got_full_map 103, nothing requested 2026-02-11T23:30:50.261+0000 7fac626e8640 10 osd.0 102 handle_osd_map got inc map for epoch 104 2026-02-11T23:30:50.261+0000 7fac626e8640 20 osd.0 102 got_full_map 104, nothing requested 2026-02-11T23:30:50.261+0000 7fac626e8640 20 osd.0 102 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:50.261+0000 7fac626e8640 20 osd.0 102 handle_osd_map pg_num_history pg_num_history(e104 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:30:50.261+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {103={},104={}} 2026-02-11T23:30:50.261+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:50.261+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e104 maps [1~104] lci=[5,104] tlb=1) 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0 102 _committed_osd_maps 103..104 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0 102 advance to epoch 103 (<= last 104 <= newest_map 104) 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0 103 advance to epoch 104 (<= last 104 <= newest_map 104) 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0 104 consume_map version 104 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0 102 await_reserved_maps epoch:104 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0 102 await_reserved_maps done 0 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0 104 consume_map 104 -- publish done 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0 104 identify_splits_and_merges 4.0 e102 to e104 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0 104 identify_splits_and_merges 4.0 e102 to e104 pg_nums {45=1} 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0 104 identify_splits_and_merges 1.0 e102 to e104 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0 104 identify_splits_and_merges 1.0 e102 to e104 pg_nums {24=1} 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0 104 prune_pg_created 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0:0.consume_map 104 (was 102) 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 103 <= 104, requeueing 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {103=} 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0:1.consume_map 104 (was 102) 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0:2.consume_map 104 (was 102) 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0:3.consume_map 104 (was 102) 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0:4.consume_map 104 (was 102) 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0:5.consume_map 104 (was 102) 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0:6.consume_map 104 (was 102) 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0:7.consume_map 104 (was 102) 2026-02-11T23:30:50.261+0000 7fac5e983640 10 osd.0 104 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:50.261+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.261+0000 7fac5e983640 15 osd.0 104 enqueue_peering_evt 4.0 epoch_sent: 104 epoch_requested: 104 NullEvt 2026-02-11T23:30:50.261+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 MLease epoch 103 from osd.4 pg_lease(ru 41.104904175s ub 49.105007172s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e103) queued 2026-02-11T23:30:50.261+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.261+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 103 epoch_requested: 103 MLease epoch 103 from osd.4 pg_lease(ru 41.104904175s ub 49.105007172s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e103) pg 0x4856000 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 NullEvt) class_id 2 prio 255 cost 10 e104) 2026-02-11T23:30:50.261+0000 7fac5e983640 15 osd.0 104 enqueue_peering_evt 1.0 epoch_sent: 104 epoch_requested: 104 NullEvt 2026-02-11T23:30:50.261+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 NullEvt) class_id 2 prio 255 cost 10 e104) 2026-02-11T23:30:50.261+0000 7fac5e983640 7 osd.0 104 activate_map version 104 2026-02-11T23:30:50.261+0000 7fac51168640 10 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 103 2026-02-11T23:30:50.261+0000 7fac55971640 20 osd.0 104 agent_entry empty queue 2026-02-11T23:30:50.261+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.261+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 NullEvt) class_id 2immediate prio 255 cost 10 e104) queued 2026-02-11T23:30:50.261+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 102 -> 103 2026-02-11T23:30:50.261+0000 7fac51168640 10 osd.0 pg_epoch: 102 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:50.261+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 104 advance_pg new pool opts old pool opts 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 104 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 103 -> 104 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 104 advance_pg new pool opts old pool opts 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 103..104 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 92 while current is 104 2026-02-11T23:30:50.262+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 103 epoch_requested: 103 MLease epoch 103 from osd.4 pg_lease(ru 41.104904175s ub 49.105007172s int 16.000000000s) 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease pg_lease(ru 41.104904175s ub 49.105007172s int 16.000000000s) 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.886751175s -> ru 50.991657257s 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until now 50.991657257s 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.886990547s -> ruub 58.991996765s 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::proc_lease readable_until_ub now 58.991996765s 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 104 get_con_osd_cluster to osd.4 from_epoch 104 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 104 get_nextmap_reserved map_reservations: {104=1} 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 104 release_map epoch: 104 2026-02-11T23:30:50.262+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_lease_ack(4.0 pg_lease_ack(ruub 49.105007172s) e104/104) -- 0x2580160 con 0x3e4a000 2026-02-11T23:30:50.262+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:50.262+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.262+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 NullEvt) class_id 2immediate prio 255 cost 10 e104) pg 0x4856000 2026-02-11T23:30:50.262+0000 7fac55170640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 104 epoch_requested: 104 NullEvt 2026-02-11T23:30:50.262+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 NullEvt) class_id 2immediate prio 255 cost 10 e104) queued 2026-02-11T23:30:50.262+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 NullEvt) class_id 2immediate prio 255 cost 10 e104) pg 0x40f8000 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] handle_advance_map: 103 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 102 -> 103 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 102 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [1]/[1] -- 1/1 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [1] newacting [1] 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:50.262+0000 7fac51168640 5 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 30.046142 165 0.000545 2026-02-11T23:30:50.262+0000 7fac51168640 5 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] exit Started/ReplicaActive 30.046147 0 0.000000 2026-02-11T23:30:50.262+0000 7fac51168640 5 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] exit Started 30.175487 0 0.000000 2026-02-11T23:30:50.262+0000 7fac51168640 5 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=27 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 103 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=103 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=103 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=103 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=103 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [1] newacting [1] 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=103 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1,0] r=1 lpr=103 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 103 2026-02-11T23:30:50.262+0000 7fac51168640 15 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1] r=1 lpr=103 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(27-102 up [1, 0](1) acting [1, 0](1)) up_thru 84 up_from 9 last_epoch_clean 28 interval(27-102 up [1, 0](1) acting [1, 0](1) maybe_went_rw) : primary up 9-84 includes interval 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=27) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([27,102] all_participants={0, 1} intervals=[([27,102] acting={0, 1})]) 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [1]+[1] 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active pruub 53.052566528s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 53.052566528s (mnow 43.097969055s + 9.954595566s) 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active pruub 53.052566528s@ mbc={}] cancel_recovery 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active pruub 53.052566528s@ mbc={}] clear_recovery_state 2026-02-11T23:30:50.262+0000 7fac51168640 15 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active pruub 53.052566528s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active pruub 53.052566528s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:50.262+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active pruub 53.052566528s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active pruub 53.052566528s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active pruub 53.052566528s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:50.262+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:50.262+0000 7fac51168640 1 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 active pruub 53.052566528s@ mbc={}] PeeringState::start_peering_interval up [1,0] -> [1], acting [1,0] -> [1], acting_primary 1 -> 1, up_primary 1 -> 1, role 1 -> -1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] clear_primary_state 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] release_backoffs [1:00000000::::0,MAX) 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:50.262+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:50.262+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] agent_stop 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] on_change 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] on_change_cleanup 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] on_change 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] exit NotTrimming 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown pruub 53.052566528s@ mbc={}] enter NotTrimming 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] plpg_on_role_change 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] hit_set_clear 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] agent_stop 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 104 advance_pg new pool opts old pool opts 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] handle_advance_map: 104 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 103 -> 104 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 103 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] PeeringState::advance_map handle_advance_map [1]/[1] -- 1/1 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] state: Reset advmap 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] hit_set_clear 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] agent_stop 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 104 advance_pg new pool opts old pool opts 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] handle_activate_map: epoch range: 103..104 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954595566s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:50.262+0000 7fac51168640 5 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954512596s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] exit Reset 0.000134 2 0.000124 2026-02-11T23:30:50.262+0000 7fac51168640 5 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954512596s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] enter Started 2026-02-11T23:30:50.262+0000 7fac51168640 5 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954512596s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] enter Start 2026-02-11T23:30:50.262+0000 7fac51168640 1 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954512596s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:50.262+0000 7fac51168640 5 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954512596s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:50.262+0000 7fac51168640 5 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954512596s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] enter Started/Stray 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954512596s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 70 while current is 104 2026-02-11T23:30:50.262+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954512596s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954512596s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] do_peering_event: epoch_sent: 104 epoch_requested: 104 NullEvt 2026-02-11T23:30:50.262+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 104 get_con_osd_cluster to osd.1 from_epoch 104 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 104 get_nextmap_reserved map_reservations: {104=1} 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 104 release_map epoch: 104 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 104 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 102 -> 104 (shared) 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 104 send_incremental_map epoch range: (103, 104] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 104 add_map_inc_bl 103 574 bytes 2026-02-11T23:30:50.262+0000 7fac51168640 10 osd.0 104 add_map_inc_bl 104 574 bytes 2026-02-11T23:30:50.262+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(103..104 src has 1..104) -- 0x46eb6c0 con 0x3752000 2026-02-11T23:30:50.262+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:104 sent:104 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954512596s) ([27,102] all_participants={0, 1} intervals=[([27,102] acting={0, 1})])) e104/104) -- 0x3382a00 con 0x3752000 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.262+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.262+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.262+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.263+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 25 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 104) e104/104) ==== 158+0+0 (unknown 0 0 0) 0x3f3f680 con 0x3752000 2026-02-11T23:30:50.263+0000 7fac703fa640 15 osd.0 104 enqueue_peering_evt 1.0 epoch_sent: 104 epoch_requested: 104 MQuery 1.0 from 1 query_epoch 104 query: query(info 0'0 epoch_sent 104) 2026-02-11T23:30:50.263+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 MQuery 1.0 from 1 query_epoch 104 query: query(info 0'0 epoch_sent 104)) class_id 2 prio 255 cost 10 e104) 2026-02-11T23:30:50.263+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.263+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 MQuery 1.0 from 1 query_epoch 104 query: query(info 0'0 epoch_sent 104)) class_id 2immediate prio 255 cost 10 e104) queued 2026-02-11T23:30:50.263+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.263+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 104 epoch_requested: 104 MQuery 1.0 from 1 query_epoch 104 query: query(info 0'0 epoch_sent 104)) class_id 2immediate prio 255 cost 10 e104) pg 0x40f8000 2026-02-11T23:30:50.263+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.263+0000 7fac55170640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.954512596s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] do_peering_event: epoch_sent: 104 epoch_requested: 104 MQuery 1.0 from 1 query_epoch 104 query: query(info 0'0 epoch_sent 104) 2026-02-11T23:30:50.263+0000 7fac55170640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.953515053s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:50.263+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.263+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 26 ==== osd_map(103..104 src has 1..104) ==== 1200+0+0 (unknown 0 0 0) 0x46eb6c0 con 0x3752000 2026-02-11T23:30:50.263+0000 7fac55170640 20 osd.0 104 get_con_osd_cluster to osd.1 from_epoch 104 2026-02-11T23:30:50.263+0000 7fac55170640 20 osd.0 104 get_nextmap_reserved map_reservations: {104=1} 2026-02-11T23:30:50.263+0000 7fac606e4640 20 osd.0 104 OSD::ms_dispatch: osd_map(103..104 src has 1..104) 2026-02-11T23:30:50.263+0000 7fac55170640 20 osd.0 104 release_map epoch: 104 2026-02-11T23:30:50.263+0000 7fac606e4640 20 osd.0 104 _dispatch 0x46eb6c0 osd_map(103..104 src has 1..104) 2026-02-11T23:30:50.263+0000 7fac55170640 10 osd.0 104 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 104 is not newer than session's projected_epoch of 104 2026-02-11T23:30:50.263+0000 7fac606e4640 3 osd.0 104 handle_osd_map epochs [103,104], i have 104, src has [1,104] 2026-02-11T23:30:50.263+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:104 sent:104 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.953515053s) ([27,102] all_participants={0, 1} intervals=[([27,102] acting={0, 1})])) e104/104) -- 0x486b100 con 0x3752000 2026-02-11T23:30:50.263+0000 7fac606e4640 10 osd.0 104 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:50.263+0000 7fac606e4640 10 osd.0 104 no new maps here, dropping 2026-02-11T23:30:50.263+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.314+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 53 ==== osd_map(104..105 src has 1..105) ==== 878+0+0 (crc 0 0 0) 0x46ea000 con 0x396bc00 2026-02-11T23:30:50.314+0000 7fac626e8640 20 osd.0 104 OSD::ms_dispatch: osd_map(104..105 src has 1..105) 2026-02-11T23:30:50.314+0000 7fac626e8640 20 osd.0 104 _dispatch 0x46ea000 osd_map(104..105 src has 1..105) 2026-02-11T23:30:50.314+0000 7fac626e8640 3 osd.0 104 handle_osd_map epochs [104,105], i have 104, src has [1,105] 2026-02-11T23:30:50.314+0000 7fac626e8640 10 osd.0 104 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:50.314+0000 7fac626e8640 10 osd.0 104 handle_osd_map got inc map for epoch 105 2026-02-11T23:30:50.314+0000 7fac626e8640 20 osd.0 104 got_full_map 105, nothing requested 2026-02-11T23:30:50.314+0000 7fac626e8640 20 osd.0 104 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:50.314+0000 7fac626e8640 20 osd.0 104 handle_osd_map pg_num_history pg_num_history(e105 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:30:50.314+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {105={}} 2026-02-11T23:30:50.314+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:50.314+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e105 maps [1~105] lci=[5,105] tlb=1) 2026-02-11T23:30:50.314+0000 7fac5e983640 10 osd.0 104 _committed_osd_maps 105..105 2026-02-11T23:30:50.314+0000 7fac5e983640 10 osd.0 104 advance to epoch 105 (<= last 105 <= newest_map 105) 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0 105 consume_map version 105 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0 104 await_reserved_maps epoch:105 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0 104 await_reserved_maps done 0 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0 105 consume_map 105 -- publish done 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0 105 identify_splits_and_merges 4.0 e104 to e105 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0 105 identify_splits_and_merges 4.0 e104 to e105 pg_nums {45=1} 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0 105 identify_splits_and_merges 1.0 e104 to e105 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0 105 identify_splits_and_merges 1.0 e104 to e105 pg_nums {24=1} 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0 105 prune_pg_created 2026-02-11T23:30:50.314+0000 7fac5e983640 10 osd.0:0.consume_map 105 (was 104) 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:50.314+0000 7fac5e983640 10 osd.0:1.consume_map 105 (was 104) 2026-02-11T23:30:50.314+0000 7fac5e983640 10 osd.0:2.consume_map 105 (was 104) 2026-02-11T23:30:50.314+0000 7fac5e983640 10 osd.0:3.consume_map 105 (was 104) 2026-02-11T23:30:50.314+0000 7fac5e983640 10 osd.0:4.consume_map 105 (was 104) 2026-02-11T23:30:50.314+0000 7fac5e983640 10 osd.0:5.consume_map 105 (was 104) 2026-02-11T23:30:50.314+0000 7fac5e983640 10 osd.0:6.consume_map 105 (was 104) 2026-02-11T23:30:50.314+0000 7fac5e983640 10 osd.0:7.consume_map 105 (was 104) 2026-02-11T23:30:50.314+0000 7fac5e983640 10 osd.0 105 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:50.314+0000 7fac5e983640 15 osd.0 105 enqueue_peering_evt 4.0 epoch_sent: 105 epoch_requested: 105 NullEvt 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2 prio 255 cost 10 e105) 2026-02-11T23:30:50.314+0000 7fac5e983640 15 osd.0 105 enqueue_peering_evt 1.0 epoch_sent: 105 epoch_requested: 105 NullEvt 2026-02-11T23:30:50.314+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2 prio 255 cost 10 e105) 2026-02-11T23:30:50.314+0000 7fac5e983640 7 osd.0 105 activate_map version 105 2026-02-11T23:30:50.314+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.314+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2immediate prio 255 cost 10 e105) queued 2026-02-11T23:30:50.314+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.314+0000 7fac55971640 20 osd.0 105 agent_entry empty queue 2026-02-11T23:30:50.314+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2immediate prio 255 cost 10 e105) pg 0x4856000 2026-02-11T23:30:50.314+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.314+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2immediate prio 255 cost 10 e105) queued 2026-02-11T23:30:50.314+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.314+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 NullEvt) class_id 2immediate prio 255 cost 10 e105) pg 0x40f8000 2026-02-11T23:30:50.314+0000 7fac55170640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 105 2026-02-11T23:30:50.314+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 104 -> 105 2026-02-11T23:30:50.314+0000 7fac55170640 10 osd.0 pg_epoch: 104 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:30:50.314+0000 7fac55170640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:50.314+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.953515053s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] handle_advance_map: 105 2026-02-11T23:30:50.314+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 104 -> 105 2026-02-11T23:30:50.314+0000 7fac55170640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:50.314+0000 7fac51168640 10 osd.0 pg_epoch: 104 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.953515053s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] PeeringState::advance_map handle_advance_map [1]/[1] -- 1/1 2026-02-11T23:30:50.314+0000 7fac55170640 20 osd.0 105 advance_pg new pool opts old pool opts 2026-02-11T23:30:50.314+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.953515053s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] state: Started advmap 2026-02-11T23:30:50.314+0000 7fac55170640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_activate_map: epoch range: 105..105 2026-02-11T23:30:50.314+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.953515053s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:50.314+0000 7fac55170640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map 2026-02-11T23:30:50.314+0000 7fac51168640 20 osd.0 105 advance_pg new pool opts old pool opts 2026-02-11T23:30:50.314+0000 7fac55170640 20 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 92 while current is 105 2026-02-11T23:30:50.314+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.314+0000 7fac55170640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] do_peering_event: epoch_sent: 105 epoch_requested: 105 NullEvt 2026-02-11T23:30:50.314+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.953515053s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] handle_activate_map: epoch range: 105..105 2026-02-11T23:30:50.314+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.314+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.314+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.953515053s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:50.314+0000 7fac51168640 20 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 104 while current is 105 2026-02-11T23:30:50.314+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:50.314+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] do_peering_event: epoch_sent: 105 epoch_requested: 105 NullEvt 2026-02-11T23:30:50.314+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:50.314+0000 7fac51168640 20 osd.0 105 get_con_osd_cluster to osd.1 from_epoch 105 2026-02-11T23:30:50.314+0000 7fac51168640 20 osd.0 105 get_nextmap_reserved map_reservations: {105=1} 2026-02-11T23:30:50.314+0000 7fac51168640 20 osd.0 105 release_map epoch: 105 2026-02-11T23:30:50.314+0000 7fac51168640 10 osd.0 105 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 104 -> 105 (shared) 2026-02-11T23:30:50.314+0000 7fac51168640 10 osd.0 105 send_incremental_map epoch range: (105, 105] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:30:50.314+0000 7fac51168640 10 osd.0 105 add_map_inc_bl 105 252 bytes 2026-02-11T23:30:50.314+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(105..105 src has 1..105) -- 0x38e3880 con 0x3752000 2026-02-11T23:30:50.314+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:105 sent:105 1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) ([27,102] all_participants={0, 1} intervals=[([27,102] acting={0, 1})])) e105/105) -- 0x4869c00 con 0x3752000 2026-02-11T23:30:50.314+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.315+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 27 ==== osd pg remove(epoch 105; pg1.0; ) ==== 32+0+0 (unknown 0 0 0) 0x2580160 con 0x3752000 2026-02-11T23:30:50.315+0000 7fac703fa640 7 osd.0 105 handle_fast_pg_remove osd pg remove(epoch 105; pg1.0; ) from osd.1 2026-02-11T23:30:50.315+0000 7fac703fa640 15 osd.0 105 enqueue_peering_evt 1.0 epoch_sent: 105 epoch_requested: 105 DeleteStart 2026-02-11T23:30:50.315+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 DeleteStart) class_id 2 prio 255 cost 10 e105) 2026-02-11T23:30:50.315+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.315+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 DeleteStart) class_id 2immediate prio 255 cost 10 e105) queued 2026-02-11T23:30:50.315+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.315+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 DeleteStart) class_id 2immediate prio 255 cost 10 e105) pg 0x40f8000 2026-02-11T23:30:50.315+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.315+0000 7fac55170640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] do_peering_event: epoch_sent: 105 epoch_requested: 105 DeleteStart 2026-02-11T23:30:50.315+0000 7fac55170640 5 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] exit Started/Stray 0.053459 6 0.000037 2026-02-11T23:30:50.315+0000 7fac55170640 5 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] enter Started/ToDelete 2026-02-11T23:30:50.315+0000 7fac55170640 5 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] enter Started/ToDelete/WaitDeleteReserved 2026-02-11T23:30:50.315+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.315+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.315+0000 7fac58176640 15 osd.0 105 enqueue_peering_evt 1.0 epoch_sent: 105 epoch_requested: 105 DeleteReserved 2026-02-11T23:30:50.315+0000 7fac58176640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 DeleteReserved) class_id 2 prio 255 cost 10 e105) 2026-02-11T23:30:50.315+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.315+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 DeleteReserved) class_id 2immediate prio 255 cost 10 e105) queued 2026-02-11T23:30:50.315+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.315+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 105 epoch_requested: 105 DeleteReserved) class_id 2immediate prio 255 cost 10 e105) pg 0x40f8000 2026-02-11T23:30:50.315+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] do_peering_event: epoch_sent: 105 epoch_requested: 105 DeleteReserved 2026-02-11T23:30:50.315+0000 7fac51168640 5 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] exit Started/ToDelete/WaitDeleteReserved 0.000032 1 0.000017 2026-02-11T23:30:50.315+0000 7fac51168640 5 osd.0 pg_epoch: 105 pg[1.0( empty local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] enter Started/ToDelete/Deleting 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] on_removal 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] on_shutdown 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:50.315+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:50.315+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] clear_backoffs 2026-02-11T23:30:50.315+0000 7fac51168640 20 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] exit NotTrimming 2026-02-11T23:30:50.315+0000 7fac51168640 20 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] enter NotTrimming 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] on_change 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] clear_async_reads 2026-02-11T23:30:50.315+0000 7fac51168640 20 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] clear_primary_state 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] release_backoffs [1:00000000::::0,MAX) 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:50.315+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:50.315+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:50.315+0000 7fac51168640 20 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] agent_stop 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] cancel_recovery 2026-02-11T23:30:50.315+0000 7fac51168640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] clear_recovery_state 2026-02-11T23:30:50.315+0000 7fac51168640 15 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:50.315+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:50.315+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.316+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.316+0000 7fac55170640 10 osd.0 105 queue_for_pg_delete on 1.0 e 105 2026-02-11T23:30:50.316+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGDelete(pgid=1.0 epoch_queued=105) class_id 1 prio 5 cost 1048576 e105) 2026-02-11T23:30:50.316+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.316+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGDelete(1.0 e105) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e105) queued 2026-02-11T23:30:50.316+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.316+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGDelete(1.0 e105) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e105) pg 0x40f8000 2026-02-11T23:30:50.316+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.316+0000 7fac55170640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] do_peering_event: epoch_sent: 105 epoch_requested: 105 DeleteSome 2026-02-11T23:30:50.316+0000 7fac55170640 10 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] do_delete_work 2026-02-11T23:30:50.316+0000 7fac55170640 20 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] do_delete_work [#1:00000000::::head#] 2026-02-11T23:30:50.316+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.316+0000 7fac55170640 20 osd.0 105 try_finish_pg_delete 1.0 0x40f8000 2026-02-11T23:30:50.316+0000 7fac55170640 10 osd.0:0._detach_pg 1.0 0x40f8000 2026-02-11T23:30:50.316+0000 7fac55170640 5 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 DELETING pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] exit Started/ToDelete/Deleting 0.000410 1 0.000034 2026-02-11T23:30:50.316+0000 7fac55170640 5 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] exit Started/ToDelete 0.000449 0 0.000000 2026-02-11T23:30:50.316+0000 7fac55170640 5 osd.0 pg_epoch: 105 pg[1.0( empty lb MIN local-lis/les=27/28 n=0 ec=24/24 lis/c=27/27 les/c/f=28/28/0 sis=103 pruub=9.902157784s) [1] r=-1 lpr=103 pi=[27,103)/1 crt=0'0 unknown NOTIFY pruub 53.052566528s@ mbc={}] exit Started 0.053921 0 0.000000 2026-02-11T23:30:50.316+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.316+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.316+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.316+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:50.316+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:50.316+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.415+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 54 ==== osd_map(105..106 src has 1..106) ==== 878+0+0 (crc 0 0 0) 0x46eb500 con 0x396bc00 2026-02-11T23:30:50.415+0000 7fac626e8640 20 osd.0 105 OSD::ms_dispatch: osd_map(105..106 src has 1..106) 2026-02-11T23:30:50.415+0000 7fac626e8640 20 osd.0 105 _dispatch 0x46eb500 osd_map(105..106 src has 1..106) 2026-02-11T23:30:50.415+0000 7fac626e8640 3 osd.0 105 handle_osd_map epochs [105,106], i have 105, src has [1,106] 2026-02-11T23:30:50.415+0000 7fac626e8640 10 osd.0 105 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:30:50.415+0000 7fac626e8640 10 osd.0 105 handle_osd_map got inc map for epoch 106 2026-02-11T23:30:50.415+0000 7fac626e8640 20 osd.0 105 got_full_map 106, nothing requested 2026-02-11T23:30:50.415+0000 7fac626e8640 20 osd.0 105 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:50.415+0000 7fac626e8640 20 osd.0 105 handle_osd_map pg_num_history pg_num_history(e106 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:30:50.415+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {106={}} 2026-02-11T23:30:50.415+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:50.415+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e106 maps [1~106] lci=[5,106] tlb=1) 2026-02-11T23:30:50.416+0000 7fac5e983640 10 osd.0 105 _committed_osd_maps 106..106 2026-02-11T23:30:50.416+0000 7fac5e983640 10 osd.0 105 advance to epoch 106 (<= last 106 <= newest_map 106) 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0 106 consume_map version 106 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0 105 await_reserved_maps epoch:106 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0 105 await_reserved_maps done 0 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0 106 consume_map 106 -- publish done 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0 106 identify_splits_and_merges 4.0 e105 to e106 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0 106 identify_splits_and_merges 4.0 e105 to e106 pg_nums {45=1} 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x2580dc0 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 1.0 has no pg and waiting_for_split 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0 106 prune_pg_created 2026-02-11T23:30:50.416+0000 7fac5e983640 10 osd.0:0.consume_map 106 (was 105) 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 empty, pruning 2026-02-11T23:30:50.416+0000 7fac5e983640 10 osd.0:1.consume_map 106 (was 105) 2026-02-11T23:30:50.416+0000 7fac5e983640 10 osd.0:2.consume_map 106 (was 105) 2026-02-11T23:30:50.416+0000 7fac5e983640 10 osd.0:3.consume_map 106 (was 105) 2026-02-11T23:30:50.416+0000 7fac5e983640 10 osd.0:4.consume_map 106 (was 105) 2026-02-11T23:30:50.416+0000 7fac5e983640 10 osd.0:5.consume_map 106 (was 105) 2026-02-11T23:30:50.416+0000 7fac5e983640 10 osd.0:6.consume_map 106 (was 105) 2026-02-11T23:30:50.416+0000 7fac5e983640 10 osd.0:7.consume_map 106 (was 105) 2026-02-11T23:30:50.416+0000 7fac5e983640 10 osd.0 106 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:50.416+0000 7fac5e983640 15 osd.0 106 enqueue_peering_evt 4.0 epoch_sent: 106 epoch_requested: 106 NullEvt 2026-02-11T23:30:50.416+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 NullEvt) class_id 2 prio 255 cost 10 e106) 2026-02-11T23:30:50.416+0000 7fac5e983640 7 osd.0 106 activate_map version 106 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 NullEvt) class_id 2immediate prio 255 cost 10 e106) queued 2026-02-11T23:30:50.416+0000 7fac55971640 20 osd.0 106 agent_entry empty queue 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 NullEvt) class_id 2immediate prio 255 cost 10 e106) pg 0x4856000 2026-02-11T23:30:50.416+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] handle_advance_map: 106 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 105 -> 106 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 105 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::advance_map handle_advance_map [4]/[4] -- 4/4 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: Started advmap 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4] newacting [4] 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:30:50.416+0000 7fac55170640 5 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 24.260411 125 0.000465 2026-02-11T23:30:50.416+0000 7fac55170640 5 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] exit Started/ReplicaActive 24.260415 0 0.000000 2026-02-11T23:30:50.416+0000 7fac55170640 5 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] exit Started 24.646592 0 0.000000 2026-02-11T23:30:50.416+0000 7fac55170640 5 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] enter Reset 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=48 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 106 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=106 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=106 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=106 crt=0'0 active mbc={}] state: Reset advmap 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=106 crt=0'0 active mbc={}] PeeringState::should_restart_peering new interval newup [4] newacting [4] 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=106 crt=0'0 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4,0] r=1 lpr=106 crt=0'0 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 106 2026-02-11T23:30:50.416+0000 7fac55170640 15 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4] r=1 lpr=106 crt=0'0 active mbc={}] clear_stats 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(48-105 up [4, 0](4) acting [4, 0](4)) up_thru 62 up_from 21 last_epoch_clean 50 interval(48-105 up [4, 0](4) acting [4, 0](4) maybe_went_rw) : primary up 21-62 includes interval 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=48) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active mbc={}] PeeringState::start_peering_interval noting past ([48,105] all_participants={0, 4} intervals=[([48,105] acting={0, 4})]) 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4]+[4] 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active pruub 58.991996765s@ mbc={}] PeeringState::on_new_interval prior_readable_until_ub 58.991996765s (mnow 43.251922607s + 15.740075111s) 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active pruub 58.991996765s@ mbc={}] cancel_recovery 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active pruub 58.991996765s@ mbc={}] clear_recovery_state 2026-02-11T23:30:50.416+0000 7fac55170640 15 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active pruub 58.991996765s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active pruub 58.991996765s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:50.416+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active pruub 58.991996765s@ mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active pruub 58.991996765s@ mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active pruub 58.991996765s@ mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:30:50.416+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:50.416+0000 7fac55170640 1 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 active pruub 58.991996765s@ mbc={}] PeeringState::start_peering_interval up [4,0] -> [4], acting [4,0] -> [4], acting_primary 4 -> 4, up_primary 4 -> 4, role 1 -> -1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] clear_primary_state 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] release_backoffs [4:00000000::::0,MAX) 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:50.416+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:50.416+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] agent_stop 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] on_change 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] on_change_cleanup 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] on_change 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] exit NotTrimming 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown pruub 58.991996765s@ mbc={}] enter NotTrimming 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] plpg_on_role_change 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] hit_set_clear 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] agent_stop 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 106 advance_pg new pool opts old pool opts 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] handle_activate_map: epoch range: 106..106 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740075111s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:50.416+0000 7fac55170640 5 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740025520s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] exit Reset 0.000080 1 0.000079 2026-02-11T23:30:50.416+0000 7fac55170640 5 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740025520s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] enter Started 2026-02-11T23:30:50.416+0000 7fac55170640 5 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740025520s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] enter Start 2026-02-11T23:30:50.416+0000 7fac55170640 1 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740025520s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] state: transitioning to Stray 2026-02-11T23:30:50.416+0000 7fac55170640 5 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740025520s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:30:50.416+0000 7fac55170640 5 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740025520s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] enter Started/Stray 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740025520s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 92 while current is 106 2026-02-11T23:30:50.416+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740025520s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740025520s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] do_peering_event: epoch_sent: 106 epoch_requested: 106 NullEvt 2026-02-11T23:30:50.416+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 106 get_con_osd_cluster to osd.4 from_epoch 106 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 102 -> 106 (shared) 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 106 send_incremental_map epoch range: (103, 106] to v2:127.0.0.1:6836/864727417 2026-02-11T23:30:50.416+0000 7fac55170640 10 osd.0 106 add_map_inc_bl 106 574 bytes 2026-02-11T23:30:50.416+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(103..106 src has 1..106) -- 0x3941c00 con 0x3e4a000 2026-02-11T23:30:50.416+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:106 sent:106 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740025520s) ([48,105] all_participants={0, 4} intervals=[([48,105] acting={0, 4})])) e106/106) -- 0x3383800 con 0x3e4a000 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.416+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.416+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.416+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.417+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 34 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 106) e106/106) ==== 158+0+0 (unknown 0 0 0) 0x3f3f680 con 0x3e4a000 2026-02-11T23:30:50.417+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 33 ==== osd_map(103..106 src has 1..106) ==== 2042+0+0 (unknown 0 0 0) 0x3941c00 con 0x3e4a000 2026-02-11T23:30:50.417+0000 7fac703fa640 15 osd.0 106 enqueue_peering_evt 4.0 epoch_sent: 106 epoch_requested: 106 MQuery 4.0 from 4 query_epoch 106 query: query(info 0'0 epoch_sent 106) 2026-02-11T23:30:50.417+0000 7fac606e4640 20 osd.0 106 OSD::ms_dispatch: osd_map(103..106 src has 1..106) 2026-02-11T23:30:50.417+0000 7fac606e4640 20 osd.0 106 _dispatch 0x3941c00 osd_map(103..106 src has 1..106) 2026-02-11T23:30:50.417+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 MQuery 4.0 from 4 query_epoch 106 query: query(info 0'0 epoch_sent 106)) class_id 2 prio 255 cost 10 e106) 2026-02-11T23:30:50.417+0000 7fac606e4640 3 osd.0 106 handle_osd_map epochs [103,106], i have 106, src has [1,106] 2026-02-11T23:30:50.417+0000 7fac606e4640 10 osd.0 106 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:30:50.417+0000 7fac606e4640 10 osd.0 106 no new maps here, dropping 2026-02-11T23:30:50.417+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.417+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 MQuery 4.0 from 4 query_epoch 106 query: query(info 0'0 epoch_sent 106)) class_id 2immediate prio 255 cost 10 e106) queued 2026-02-11T23:30:50.417+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.417+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 106 epoch_requested: 106 MQuery 4.0 from 4 query_epoch 106 query: query(info 0'0 epoch_sent 106)) class_id 2immediate prio 255 cost 10 e106) pg 0x4856000 2026-02-11T23:30:50.417+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.417+0000 7fac51168640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.740025520s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] do_peering_event: epoch_sent: 106 epoch_requested: 106 MQuery 4.0 from 4 query_epoch 106 query: query(info 0'0 epoch_sent 106) 2026-02-11T23:30:50.417+0000 7fac51168640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.738779068s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:30:50.417+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:50.417+0000 7fac51168640 20 osd.0 106 get_con_osd_cluster to osd.4 from_epoch 106 2026-02-11T23:30:50.417+0000 7fac51168640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:30:50.417+0000 7fac51168640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:30:50.417+0000 7fac51168640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 106 is not newer than session's projected_epoch of 106 2026-02-11T23:30:50.417+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:106 sent:106 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.738779068s) ([48,105] all_participants={0, 4} intervals=[([48,105] acting={0, 4})])) e106/106) -- 0x4868000 con 0x3e4a000 2026-02-11T23:30:50.417+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.517+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 35 ==== osd pg remove(epoch 107; pg4.0; ) ==== 32+0+0 (unknown 0 0 0) 0x2580160 con 0x3e4a000 2026-02-11T23:30:50.517+0000 7fac703fa640 7 osd.0 106 handle_fast_pg_remove osd pg remove(epoch 107; pg4.0; ) from osd.4 2026-02-11T23:30:50.517+0000 7fac703fa640 15 osd.0 106 enqueue_peering_evt 4.0 epoch_sent: 107 epoch_requested: 107 DeleteStart 2026-02-11T23:30:50.517+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 DeleteStart) class_id 2 prio 255 cost 10 e107) 2026-02-11T23:30:50.517+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.517+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 DeleteStart) class_id 2immediate prio 255 cost 10 e107) queued 2026-02-11T23:30:50.517+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.517+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 DeleteStart) class_id 2immediate prio 255 cost 10 e107) pg 0x4856000 2026-02-11T23:30:50.517+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 107, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 DeleteStart) class_id 2immediate prio 255 cost 10 e107) 2026-02-11T23:30:50.517+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.517+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.656+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 12 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:30:50.657859+0000/33.605365753s send_stamp 33.605365753s delta_ub -9.886726379s) ==== 2033+0+0 (crc 0 0 0) 0x3da2f00 con 0x3f57800 2026-02-11T23:30:50.656+0000 7fac71bfd640 20 osd.0 106 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887003899s,-9.886726379s]) 2026-02-11T23:30:50.656+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 12 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:30:50.657859+0000/33.605365753s send_stamp 33.605365753s delta_ub -9.886726379s) ==== 2033+0+0 (crc 0 0 0) 0x3da3680 con 0x3f57400 2026-02-11T23:30:50.656+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e106 up_from 5 ping_stamp 2026-02-11T23:30:50.657859+0000/33.605365753s send_stamp 43.492366791s delta_ub 9.887003899s) -- 0x3da3200 con 0x3f57800 2026-02-11T23:30:50.656+0000 7fac71bfd640 20 osd.0 106 get_con_osd_cluster to osd.4 from_epoch 106 2026-02-11T23:30:50.656+0000 7fac71bfd640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:30:50.656+0000 7fac71bfd640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:30:50.656+0000 7fac71bfd640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6836/864727417 updating session's projected_epoch from 106 to ping map epoch of 107 2026-02-11T23:30:50.656+0000 7fac71bfd640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 106 is not newer than session's projected_epoch of 107 2026-02-11T23:30:50.656+0000 7fac703fa640 20 osd.0 106 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887025833s,-9.886726379s]) 2026-02-11T23:30:50.656+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e106 up_from 5 ping_stamp 2026-02-11T23:30:50.657859+0000/33.605365753s send_stamp 43.492393494s delta_ub 9.887025833s) -- 0x3da3380 con 0x3f57400 2026-02-11T23:30:50.656+0000 7fac703fa640 20 osd.0 106 get_con_osd_cluster to osd.4 from_epoch 106 2026-02-11T23:30:50.656+0000 7fac703fa640 20 osd.0 106 get_nextmap_reserved map_reservations: {106=1} 2026-02-11T23:30:50.656+0000 7fac703fa640 20 osd.0 106 release_map epoch: 106 2026-02-11T23:30:50.656+0000 7fac703fa640 10 osd.0 106 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 106 is not newer than session's projected_epoch of 107 2026-02-11T23:30:50.656+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 36 ==== osd_map(107..107 src has 1..107) ==== 626+0+0 (unknown 0 0 0) 0x38e3880 con 0x3e4a000 2026-02-11T23:30:50.656+0000 7fac606e4640 20 osd.0 106 OSD::ms_dispatch: osd_map(107..107 src has 1..107) 2026-02-11T23:30:50.656+0000 7fac606e4640 20 osd.0 106 _dispatch 0x38e3880 osd_map(107..107 src has 1..107) 2026-02-11T23:30:50.656+0000 7fac606e4640 3 osd.0 106 handle_osd_map epochs [107,107], i have 106, src has [1,107] 2026-02-11T23:30:50.656+0000 7fac606e4640 10 osd.0 106 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:30:50.656+0000 7fac606e4640 10 osd.0 106 handle_osd_map got inc map for epoch 107 2026-02-11T23:30:50.656+0000 7fac606e4640 20 osd.0 106 got_full_map 107, nothing requested 2026-02-11T23:30:50.656+0000 7fac606e4640 20 osd.0 106 trim_maps: min=1 oldest_map=1 2026-02-11T23:30:50.656+0000 7fac606e4640 20 osd.0 106 handle_osd_map pg_num_history pg_num_history(e107 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:30:50.657+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {107={}} 2026-02-11T23:30:50.657+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:30:50.657+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e107 maps [1~107] lci=[5,107] tlb=1) 2026-02-11T23:30:50.657+0000 7fac5e983640 10 osd.0 106 _committed_osd_maps 107..107 2026-02-11T23:30:50.657+0000 7fac5e983640 10 osd.0 106 advance to epoch 107 (<= last 107 <= newest_map 107) 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0 107 consume_map version 107 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0 106 await_reserved_maps epoch:107 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0 106 await_reserved_maps done 0 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0 107 consume_map 107 -- publish done 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0 107 identify_splits_and_merges 4.0 e106 to e107 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0 107 identify_splits_and_merges 4.0 e106 to e107 pg_nums {45=1} 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0 107 prune_pg_created 2026-02-11T23:30:50.657+0000 7fac5e983640 10 osd.0:0.consume_map 107 (was 106) 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 107 <= 107, requeueing 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {107=} 2026-02-11T23:30:50.657+0000 7fac5e983640 10 osd.0:1.consume_map 107 (was 106) 2026-02-11T23:30:50.657+0000 7fac5e983640 10 osd.0:2.consume_map 107 (was 106) 2026-02-11T23:30:50.657+0000 7fac5e983640 10 osd.0:3.consume_map 107 (was 106) 2026-02-11T23:30:50.657+0000 7fac5e983640 10 osd.0:4.consume_map 107 (was 106) 2026-02-11T23:30:50.657+0000 7fac5e983640 10 osd.0:5.consume_map 107 (was 106) 2026-02-11T23:30:50.657+0000 7fac5e983640 10 osd.0:6.consume_map 107 (was 106) 2026-02-11T23:30:50.657+0000 7fac5e983640 10 osd.0:7.consume_map 107 (was 106) 2026-02-11T23:30:50.657+0000 7fac5e983640 10 osd.0 107 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 DeleteStart) class_id 2immediate prio 255 cost 10 e107) queued 2026-02-11T23:30:50.657+0000 7fac5e983640 15 osd.0 107 enqueue_peering_evt 4.0 epoch_sent: 107 epoch_requested: 107 NullEvt 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 DeleteStart) class_id 2immediate prio 255 cost 10 e107) pg 0x4856000 2026-02-11T23:30:50.657+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 NullEvt) class_id 2 prio 255 cost 10 e107) 2026-02-11T23:30:50.657+0000 7fac5e983640 7 osd.0 107 activate_map version 107 2026-02-11T23:30:50.657+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.657+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 NullEvt) class_id 2immediate prio 255 cost 10 e107) queued 2026-02-11T23:30:50.657+0000 7fac55971640 20 osd.0 107 agent_entry empty queue 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.738779068s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] handle_advance_map: 107 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 106 -> 107 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 106 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.738779068s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] PeeringState::advance_map handle_advance_map [4]/[4] -- 4/4 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.738779068s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] state: Started advmap 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.738779068s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.738779068s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] plpg_on_pool_change 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.738779068s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] hit_set_clear 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.738779068s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] agent_stop 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 107 advance_pg new pool opts old pool opts 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.738779068s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] handle_activate_map: epoch range: 107..107 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.738779068s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] PeeringState::activate_map 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 106 while current is 107 2026-02-11T23:30:50.657+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] do_peering_event: epoch_sent: 107 epoch_requested: 107 DeleteStart 2026-02-11T23:30:50.657+0000 7fac55170640 5 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] exit Started/Stray 0.241220 5 0.000033 2026-02-11T23:30:50.657+0000 7fac55170640 5 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] enter Started/ToDelete 2026-02-11T23:30:50.657+0000 7fac55170640 5 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] enter Started/ToDelete/WaitDeleteReserved 2026-02-11T23:30:50.657+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:50.657+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:107 sent:107 4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) ([48,105] all_participants={0, 4} intervals=[([48,105] acting={0, 4})])) e107/107) -- 0x3dc2000 con 0x3e4a000 2026-02-11T23:30:50.657+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.657+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 NullEvt) class_id 2immediate prio 255 cost 10 e107) pg 0x4856000 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.657+0000 7fac51168640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] do_peering_event: epoch_sent: 107 epoch_requested: 107 NullEvt 2026-02-11T23:30:50.657+0000 7fac51168640 10 log is not dirty 2026-02-11T23:30:50.657+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.657+0000 7fac58176640 15 osd.0 107 enqueue_peering_evt 4.0 epoch_sent: 107 epoch_requested: 107 DeleteReserved 2026-02-11T23:30:50.657+0000 7fac58176640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 DeleteReserved) class_id 2 prio 255 cost 10 e107) 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 DeleteReserved) class_id 2immediate prio 255 cost 10 e107) queued 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 107 epoch_requested: 107 DeleteReserved) class_id 2immediate prio 255 cost 10 e107) pg 0x4856000 2026-02-11T23:30:50.657+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] do_peering_event: epoch_sent: 107 epoch_requested: 107 DeleteReserved 2026-02-11T23:30:50.657+0000 7fac55170640 5 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] exit Started/ToDelete/WaitDeleteReserved 0.000058 2 0.000014 2026-02-11T23:30:50.657+0000 7fac55170640 5 osd.0 pg_epoch: 107 pg[4.0( empty local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] enter Started/ToDelete/Deleting 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] on_removal 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] on_shutdown 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:50.657+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:50.657+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] cancel_copy_ops 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] cancel_flush_ops 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] cancel_proxy_ops 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] cancel_manifest_ops 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] cancel_cls_gather_ops 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] clear_backoffs 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] exit NotTrimming 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] enter NotTrimming 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] on_change 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] clear_async_reads 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] clear_primary_state 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] release_backoffs [4:00000000::::0,MAX) 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:30:50.657+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:50.657+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] agent_stop 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] cancel_recovery 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] clear_recovery_state 2026-02-11T23:30:50.657+0000 7fac55170640 15 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] _clear_recovery_state 2026-02-11T23:30:50.657+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 107 queue_for_pg_delete on 4.0 e 107 2026-02-11T23:30:50.657+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGDelete(pgid=4.0 epoch_queued=107) class_id 1 prio 5 cost 1048576 e107) 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGDelete(4.0 e107) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e107) queued 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGDelete(4.0 e107) class_id 1background_best_effort prio 5 qos_cost 1048576 cost 1048576 e107) pg 0x4856000 2026-02-11T23:30:50.657+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] do_peering_event: epoch_sent: 107 epoch_requested: 107 DeleteSome 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] do_delete_work 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] do_delete_work [#4:00000000::::head#] 2026-02-11T23:30:50.657+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 107 try_finish_pg_delete 4.0 0x4856000 2026-02-11T23:30:50.657+0000 7fac55170640 10 osd.0:0._detach_pg 4.0 0x4856000 2026-02-11T23:30:50.657+0000 7fac55170640 5 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 DELETING pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] exit Started/ToDelete/Deleting 0.000465 1 0.000046 2026-02-11T23:30:50.657+0000 7fac55170640 5 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] exit Started/ToDelete 0.000530 0 0.000000 2026-02-11T23:30:50.657+0000 7fac55170640 5 osd.0 pg_epoch: 107 pg[4.0( empty lb MIN local-lis/les=48/49 n=0 ec=45/45 lis/c=48/48 les/c/f=49/50/0 sis=106 pruub=15.498813629s) [4] r=-1 lpr=106 pi=[48,106)/1 crt=0'0 unknown NOTIFY pruub 58.991996765s@ mbc={}] exit Started 0.241760 0 0.000000 2026-02-11T23:30:50.657+0000 7fac55170640 10 log is not dirty 2026-02-11T23:30:50.657+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.657+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.657+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:30:50.657+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:30:50.658+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:30:50.891+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> conn(0x48e6c00 0x40ebb80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:30:50.891+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> conn(0x48e6c00 0x40ebb80 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:30:50.891+0000 7fac71bfd640 10 osd.0 107 ms_handle_fast_authentication new session 0x3940540 con 0x48e6c00 entity client.admin addr 2026-02-11T23:30:50.891+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/3505930894 conn(0x48e6c00 0x40ebb80 crc :-1 s=READY pgs=3 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=client.4297 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:30:50.891+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.4297 127.0.0.1:0/3505930894 1 ==== command(tid 1: {"prefix": "get_command_descriptions"}) ==== 62+0+0 (crc 0 0 0) 0x25d9340 con 0x48e6c00 2026-02-11T23:30:50.891+0000 7fac626e8640 20 osd.0 107 OSD::ms_dispatch: command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:30:50.891+0000 7fac626e8640 20 osd.0 107 _dispatch 0x25d9340 command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:30:50.891+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/3505930894 -- command_reply(tid 1: 0 ) -- 0x25d94a0 con 0x48e6c00 2026-02-11T23:30:50.899+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.4297 127.0.0.1:0/3505930894 2 ==== command(tid 2: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x25d94a0 con 0x48e6c00 2026-02-11T23:30:50.899+0000 7fac626e8640 20 osd.0 107 OSD::ms_dispatch: command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:50.899+0000 7fac626e8640 20 osd.0 107 _dispatch 0x25d94a0 command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:50.899+0000 7fac713fc640 15 osd.0 107 collect_pg_stats 2026-02-11T23:30:50.899+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836492 v 0) -- 0x3d9e300 con 0x396a800 2026-02-11T23:30:50.899+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/3505930894 -- command_reply(tid 2: 0 ) -- 0x25d9340 con 0x48e6c00 2026-02-11T23:30:50.905+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.4297 127.0.0.1:0/3505930894 3 ==== command(tid 3: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x25d9340 con 0x48e6c00 2026-02-11T23:30:50.905+0000 7fac626e8640 20 osd.0 107 OSD::ms_dispatch: command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:50.905+0000 7fac626e8640 20 osd.0 107 _dispatch 0x25d9340 command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:30:50.905+0000 7fac713fc640 15 osd.0 107 collect_pg_stats 2026-02-11T23:30:50.905+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836493 v 0) -- 0x3df0000 con 0x396a800 2026-02-11T23:30:50.905+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/3505930894 -- command_reply(tid 3: 0 ) -- 0x25d94a0 con 0x48e6c00 2026-02-11T23:30:50.905+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/3505930894 conn(0x48e6c00 msgr2=0x40ebb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 79 2026-02-11T23:30:50.905+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/3505930894 conn(0x48e6c00 msgr2=0x40ebb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-11T23:30:50.905+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/3505930894 conn(0x48e6c00 0x40ebb80 crc :-1 s=READY pgs=3 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-11T23:30:50.905+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/3505930894 conn(0x48e6c00 0x40ebb80 crc :-1 s=READY pgs=3 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:30:50.905+0000 7fac626e8640 2 osd.0 107 ms_handle_reset con 0x48e6c00 session 0x3940540 2026-02-11T23:30:51.123+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:30:51.123+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:51.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:51.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:51.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:21.169698+0000) 2026-02-11T23:30:51.254+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:30:51.254+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:51.254+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:51.254+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:51.254+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:51.254+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.107 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:51.254+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:51.255, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:51.254+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:51.254+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:51.254+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:51.254+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:51.254+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:51.757+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 13 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:30:51.758059+0000/34.705566406s send_stamp 34.705566406s delta_ub -9.886727333s) ==== 2033+0+0 (crc 0 0 0) 0x3da3200 con 0x3f57800 2026-02-11T23:30:51.757+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 13 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:30:51.758059+0000/34.705566406s send_stamp 34.705566406s delta_ub -9.886727333s) ==== 2033+0+0 (crc 0 0 0) 0x3da3380 con 0x3f57400 2026-02-11T23:30:51.757+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887059212s,-9.886727333s]) 2026-02-11T23:30:51.757+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:51.758059+0000/34.705566406s send_stamp 44.592624664s delta_ub 9.887059212s) -- 0x3da2f00 con 0x3f57800 2026-02-11T23:30:51.757+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:51.757+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:51.757+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:51.757+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:51.757+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887091637s,-9.886727333s]) 2026-02-11T23:30:51.757+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:51.758059+0000/34.705566406s send_stamp 44.592655182s delta_ub 9.887091637s) -- 0x3da3680 con 0x3f57400 2026-02-11T23:30:51.757+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:51.757+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:51.757+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:51.757+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:52.135+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:30:52.135+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:52.169+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:52.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:52.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:22.169765+0000) 2026-02-11T23:30:52.225+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:30:52.225+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:52.225+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:52.225+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:52.225+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:52.225+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.107 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:52.225+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:52.226, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:52.225+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:52.225+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:52.225+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:52.225+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:52.225+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:52.270+0000 7fac4d160640 5 osd.0 107 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7ca37/0x118000, compress 0x0/0x0/0x0, omap 0xa68e, meta 0x1a45972), peers [1,2,3,4] op hist []) 2026-02-11T23:30:52.270+0000 7fac4d160640 20 osd.0 107 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:52.270+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 45.106010437s delta_ub 1.980962873s) -- 0x3da2300 con 0x396b400 2026-02-11T23:30:52.270+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 45.106010437s delta_ub 1.980962873s) -- 0x3da2000 con 0x3752400 2026-02-11T23:30:52.270+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 45.106010437s delta_ub 4.649079323s) -- 0x3da2780 con 0x396b800 2026-02-11T23:30:52.270+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 45.106010437s delta_ub 4.649079323s) -- 0x3da2180 con 0x34c1000 2026-02-11T23:30:52.270+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 45.106010437s delta_ub 7.310455799s) -- 0x3da2a80 con 0x3740000 2026-02-11T23:30:52.270+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 45.106010437s delta_ub 7.310455799s) -- 0x3da2900 con 0x3740400 2026-02-11T23:30:52.270+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 45.106010437s delta_ub 9.887091637s) -- 0x3da2c00 con 0x3740800 2026-02-11T23:30:52.270+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 45.106010437s delta_ub 9.887091637s) -- 0x3b1a600 con 0x3740c00 2026-02-11T23:30:52.270+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 16 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 43.125232697s delta_ub -1.980780363s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a600 con 0x396b400 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:52.270+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 105 to ping map epoch of 107 2026-02-11T23:30:52.270+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980924726s,-1.980780363s]) 2026-02-11T23:30:52.270+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 16 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 43.125259399s delta_ub -1.980747938s) ==== 2033+0+0 (crc 0 0 0) 0x3da2180 con 0x3752400 2026-02-11T23:30:52.270+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:30:52.270+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:52.270+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:52.270+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:52.270+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 13 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 37.795738220s delta_ub -7.310272217s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a600 con 0x3740000 2026-02-11T23:30:52.270+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980905056s,-1.980747938s]) 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:52.270+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 102 to ping map epoch of 107 2026-02-11T23:30:52.270+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310444355s,-7.310272217s]) 2026-02-11T23:30:52.270+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 15 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 40.457103729s delta_ub -4.648905754s) ==== 2033+0+0 (crc 0 0 0) 0x3da2180 con 0x34c1000 2026-02-11T23:30:52.270+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:52.270+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:52.270+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:52.270+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 102 to ping map epoch of 107 2026-02-11T23:30:52.270+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:52.270+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 13 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 35.219238281s delta_ub -9.886771202s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a600 con 0x3740c00 2026-02-11T23:30:52.270+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649088383s,-4.648905754s]) 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:52.270+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:52.270+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886966705s,-9.886771202s]) 2026-02-11T23:30:52.270+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 15 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 40.457134247s delta_ub -4.648878098s) ==== 2033+0+0 (crc 0 0 0) 0x3da2c00 con 0x396b800 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:52.270+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649094582s,-4.648878098s]) 2026-02-11T23:30:52.270+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 13 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 37.795780182s delta_ub -7.310230732s) ==== 2033+0+0 (crc 0 0 0) 0x3da2c00 con 0x3740400 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:52.270+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310491562s,-7.310230732s]) 2026-02-11T23:30:52.270+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 13 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:30:52.271594+0000/45.106010437s send_stamp 35.219272614s delta_ub -9.886739731s) ==== 2033+0+0 (crc 0 0 0) 0x3da2c00 con 0x3740800 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:52.270+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:52.270+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887024879s,-9.886739731s]) 2026-02-11T23:30:52.318+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 9 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:30:52.318827+0000/40.504222870s send_stamp 40.504222870s delta_ub -4.648878098s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a600 con 0x3f56400 2026-02-11T23:30:52.318+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649101734s,-4.648878098s]) 2026-02-11T23:30:52.318+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:52.318827+0000/40.504222870s send_stamp 45.153324127s delta_ub 4.649101734s) -- 0x3da2a80 con 0x3f56400 2026-02-11T23:30:52.318+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 9 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:30:52.318827+0000/40.504222870s send_stamp 40.504222870s delta_ub -4.648878098s) ==== 2033+0+0 (crc 0 0 0) 0x3da2c00 con 0x3f56000 2026-02-11T23:30:52.318+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:52.318+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:52.318+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:52.318+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:52.318+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649120331s,-4.648878098s]) 2026-02-11T23:30:52.318+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:52.318827+0000/40.504222870s send_stamp 45.153343201s delta_ub 4.649120331s) -- 0x3da3800 con 0x3f56000 2026-02-11T23:30:52.318+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:52.318+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:52.318+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:52.318+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:53.011+0000 7fac60ee5640 20 osd.0 107 reports for 0 queries 2026-02-11T23:30:53.011+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x389ca80 con 0x396a800 2026-02-11T23:30:53.011+0000 7fac60ee5640 15 osd.0 107 collect_pg_stats 2026-02-11T23:30:53.011+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836494 v 0) -- 0x3d9ec00 con 0x396a800 2026-02-11T23:30:53.129+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:30:53.129+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:53.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:53.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:53.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:23.169825+0000) 2026-02-11T23:30:53.206+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:30:53.206+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:53.206+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:53.206+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:53.206+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:53.206+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.107 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:53.206+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:53.208, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:53.206+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:53.206+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:53.206+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:53.206+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:53.206+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:54.097+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:30:54.097+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:54.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:54.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:54.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:24.169871+0000) 2026-02-11T23:30:54.172+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:30:54.172+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:54.172+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:54.172+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:54.172+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:54.172+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.107 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:54.172+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:54.174, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:54.172+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:54.172+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:54.172+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:54.172+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:54.172+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:54.656+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 14 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:30:54.658266+0000/37.605773926s send_stamp 37.605773926s delta_ub -9.886739731s) ==== 2033+0+0 (crc 0 0 0) 0x3da2a80 con 0x3f57400 2026-02-11T23:30:54.656+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 14 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:30:54.658266+0000/37.605773926s send_stamp 37.605773926s delta_ub -9.886739731s) ==== 2033+0+0 (crc 0 0 0) 0x3da2180 con 0x3f57800 2026-02-11T23:30:54.656+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886981964s,-9.886739731s]) 2026-02-11T23:30:54.656+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:54.658266+0000/37.605773926s send_stamp 47.492755890s delta_ub 9.886981964s) -- 0x3b1a600 con 0x3f57400 2026-02-11T23:30:54.656+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:54.656+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:54.656+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:54.656+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:54.656+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887009621s,-9.886739731s]) 2026-02-11T23:30:54.656+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:54.658266+0000/37.605773926s send_stamp 47.492782593s delta_ub 9.887009621s) -- 0x3da2000 con 0x3f57800 2026-02-11T23:30:54.656+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:54.656+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:54.656+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:54.656+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:54.779+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 11 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:30:54.781193+0000/40.305229187s send_stamp 40.305229187s delta_ub -7.310230732s) ==== 2033+0+0 (crc 0 0 0) 0x3da3800 con 0x3f56c00 2026-02-11T23:30:54.779+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 11 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:30:54.781193+0000/40.305229187s send_stamp 40.305229187s delta_ub -7.310230732s) ==== 2033+0+0 (crc 0 0 0) 0x3da2000 con 0x3f56800 2026-02-11T23:30:54.779+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310465813s,-7.310230732s]) 2026-02-11T23:30:54.779+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:54.781193+0000/40.305229187s send_stamp 47.615695953s delta_ub 7.310465813s) -- 0x3da2c00 con 0x3f56c00 2026-02-11T23:30:54.779+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:30:54.779+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:54.779+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:54.779+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:54.779+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310494423s,-7.310230732s]) 2026-02-11T23:30:54.779+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:54.781193+0000/40.305229187s send_stamp 47.615722656s delta_ub 7.310494423s) -- 0x3da2180 con 0x3f56800 2026-02-11T23:30:54.779+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:30:54.779+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:54.779+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:54.779+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.089+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:30:55.089+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:55.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:55.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:55.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:25.169949+0000) 2026-02-11T23:30:55.196+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:30:55.196+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:55.196+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:55.196+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:55.196+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:55.196+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.102 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:55.196+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:55.197, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:55.196+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:55.196+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:55.196+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:55.196+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:55.196+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:55.217+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 10 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:30:55.219091+0000/43.404483795s send_stamp 43.404483795s delta_ub -4.648934364s) ==== 2033+0+0 (crc 0 0 0) 0x3da2c00 con 0x3f56000 2026-02-11T23:30:55.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 10 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:30:55.219091+0000/43.404483795s send_stamp 43.404483795s delta_ub -4.648934364s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a600 con 0x3f56400 2026-02-11T23:30:55.217+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649105072s,-4.648934364s]) 2026-02-11T23:30:55.217+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:55.219091+0000/43.404483795s send_stamp 48.053592682s delta_ub 4.649105072s) -- 0x3da3800 con 0x3f56000 2026-02-11T23:30:55.217+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:55.217+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.217+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.217+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.217+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649125576s,-4.648934364s]) 2026-02-11T23:30:55.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:55.219091+0000/43.404483795s send_stamp 48.053611755s delta_ub 4.649125576s) -- 0x3da2a80 con 0x3f56400 2026-02-11T23:30:55.217+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:55.217+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.217+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.217+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.749+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 14 ==== osd_ping(ping e107 up_from 9 ping_stamp 2026-02-11T23:30:55.751314+0000/46.604877472s send_stamp 46.604877472s delta_ub -1.980747938s) ==== 2033+0+0 (crc 0 0 0) 0x3da2180 con 0x396a400 2026-02-11T23:30:55.749+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 14 ==== osd_ping(ping e107 up_from 9 ping_stamp 2026-02-11T23:30:55.751314+0000/46.604877472s send_stamp 46.604877472s delta_ub -1.980747938s) ==== 2033+0+0 (crc 0 0 0) 0x3da3800 con 0x396a000 2026-02-11T23:30:55.750+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980944514s,-1.980747938s]) 2026-02-11T23:30:55.750+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:55.751314+0000/46.604877472s send_stamp 48.585823059s delta_ub 1.980944514s) -- 0x3da2c00 con 0x396a000 2026-02-11T23:30:55.750+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:30:55.750+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.750+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.750+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.750+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980976105s,-1.980747938s]) 2026-02-11T23:30:55.750+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:55.751314+0000/46.604877472s send_stamp 48.585853577s delta_ub 1.980976105s) -- 0x3da2000 con 0x396a400 2026-02-11T23:30:55.750+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:30:55.750+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.750+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.750+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.770+0000 7fac4d160640 5 osd.0 107 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7ca37/0x118000, compress 0x0/0x0/0x0, omap 0xa68e, meta 0x1a45972), peers [1,2,3,4] op hist []) 2026-02-11T23:30:55.770+0000 7fac4d160640 20 osd.0 107 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:55.770+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 48.606216431s delta_ub 1.980976105s) -- 0x3b1b980 con 0x396b400 2026-02-11T23:30:55.770+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 48.606216431s delta_ub 1.980976105s) -- 0x3b1bc80 con 0x3752400 2026-02-11T23:30:55.770+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 48.606216431s delta_ub 4.649125576s) -- 0x3b1bb00 con 0x396b800 2026-02-11T23:30:55.770+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 48.606216431s delta_ub 4.649125576s) -- 0x3b1b080 con 0x34c1000 2026-02-11T23:30:55.770+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 48.606216431s delta_ub 7.310494423s) -- 0x3b1a780 con 0x3740000 2026-02-11T23:30:55.770+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 48.606216431s delta_ub 7.310494423s) -- 0x3b1a300 con 0x3740400 2026-02-11T23:30:55.770+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 48.606216431s delta_ub 9.887009621s) -- 0x3da2900 con 0x3740800 2026-02-11T23:30:55.770+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 48.606216431s delta_ub 9.887009621s) -- 0x3da2780 con 0x3740c00 2026-02-11T23:30:55.770+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 17 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 46.625427246s delta_ub -1.980788827s) ==== 2033+0+0 (crc 0 0 0) 0x3da2780 con 0x396b400 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.770+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980903387s,-1.980788827s]) 2026-02-11T23:30:55.770+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 16 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 43.957267761s delta_ub -4.648944855s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b080 con 0x34c1000 2026-02-11T23:30:55.770+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:55.770+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.770+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.770+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 14 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 41.295932770s delta_ub -7.310281754s) ==== 2033+0+0 (crc 0 0 0) 0x3da2780 con 0x3740000 2026-02-11T23:30:55.770+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.770+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649067879s,-4.648944855s]) 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.770+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310416222s,-7.310281754s]) 2026-02-11T23:30:55.770+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 17 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 46.625438690s delta_ub -1.980773091s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b080 con 0x3752400 2026-02-11T23:30:55.770+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:30:55.770+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.770+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.770+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.770+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980921268s,-1.980773091s]) 2026-02-11T23:30:55.770+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 16 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 43.957283020s delta_ub -4.648929596s) ==== 2033+0+0 (crc 0 0 0) 0x3da2900 con 0x396b800 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.770+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649108410s,-4.648929596s]) 2026-02-11T23:30:55.770+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 14 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 38.719425201s delta_ub -9.886788368s) ==== 2033+0+0 (crc 0 0 0) 0x3da2780 con 0x3740c00 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.770+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.770+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886980057s,-9.886788368s]) 2026-02-11T23:30:55.770+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 14 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 41.295967102s delta_ub -7.310247421s) ==== 2033+0+0 (crc 0 0 0) 0x3da2900 con 0x3740400 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.770+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310451508s,-7.310247421s]) 2026-02-11T23:30:55.770+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 14 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:30:55.771798+0000/48.606216431s send_stamp 38.719474792s delta_ub -9.886738777s) ==== 2033+0+0 (crc 0 0 0) 0x3da2900 con 0x3740800 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:55.770+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:55.770+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886960983s,-9.886738777s]) 2026-02-11T23:30:56.081+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:30:56.081+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:56.150+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:30:56.150+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:56.150+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:56.150+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.102 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:56.150+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:56.151, recovery is active?:false restrictions:<....> 2026-02-11T23:30:56.150+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:56.150+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:56.150+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:56.150+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:56.150+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:56.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:56.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:56.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:26.170025+0000) 2026-02-11T23:30:56.317+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 11 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:30:56.319294+0000/44.504692078s send_stamp 44.504692078s delta_ub -4.648929596s) ==== 2033+0+0 (crc 0 0 0) 0x3da2780 con 0x3f56400 2026-02-11T23:30:56.317+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 11 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:30:56.319294+0000/44.504692078s send_stamp 44.504692078s delta_ub -4.648929596s) ==== 2033+0+0 (crc 0 0 0) 0x3da2900 con 0x3f56000 2026-02-11T23:30:56.317+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649115086s,-4.648929596s]) 2026-02-11T23:30:56.317+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:56.319294+0000/44.504692078s send_stamp 49.153804779s delta_ub 4.649115086s) -- 0x3b1a780 con 0x3f56400 2026-02-11T23:30:56.317+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:56.317+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:56.317+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:56.318+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:56.318+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649135590s,-4.648929596s]) 2026-02-11T23:30:56.318+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:56.319294+0000/44.504692078s send_stamp 49.153827667s delta_ub 4.649135590s) -- 0x3b1a300 con 0x3f56000 2026-02-11T23:30:56.318+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:56.318+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:56.318+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:56.318+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:57.033+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:30:57.033+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:57.114+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:30:57.114+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:57.114+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:57.114+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:57.114+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:57.114+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.102 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:57.114+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:57.115, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:57.114+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:57.114+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:57.114+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:57.114+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:57.114+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:57.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:57.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:57.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:27.170100+0000) 2026-02-11T23:30:57.329+0000 7fac6dbf5640 10 monclient: _renew_subs 2026-02-11T23:30:57.329+0000 7fac6dbf5640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:30:57.329+0000 7fac6dbf5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=108}) -- 0x38f5d40 con 0x396bc00 2026-02-11T23:30:58.008+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:30:58.008+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:58.011+0000 7fac60ee5640 20 osd.0 107 reports for 0 queries 2026-02-11T23:30:58.011+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x389d180 con 0x396a800 2026-02-11T23:30:58.011+0000 7fac60ee5640 15 osd.0 107 collect_pg_stats 2026-02-11T23:30:58.011+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836495 v 0) -- 0x3d9ef00 con 0x396a800 2026-02-11T23:30:58.018+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 12 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:30:58.019471+0000/46.204868317s send_stamp 46.204868317s delta_ub -4.648913383s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a300 con 0x3f56000 2026-02-11T23:30:58.018+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 12 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:30:58.019471+0000/46.204868317s send_stamp 46.204868317s delta_ub -4.648913383s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a780 con 0x3f56400 2026-02-11T23:30:58.018+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649097919s,-4.648913383s]) 2026-02-11T23:30:58.018+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:58.019471+0000/46.204868317s send_stamp 50.853965759s delta_ub 4.649097919s) -- 0x3da2900 con 0x3f56000 2026-02-11T23:30:58.018+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:58.018+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.018+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.018+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:58.018+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649119377s,-4.648913383s]) 2026-02-11T23:30:58.018+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:58.019471+0000/46.204868317s send_stamp 50.853984833s delta_ub 4.649119377s) -- 0x3da2780 con 0x3f56400 2026-02-11T23:30:58.018+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:58.018+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.018+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.018+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:58.070+0000 7fac4d160640 5 osd.0 107 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7ca37/0x118000, compress 0x0/0x0/0x0, omap 0xa68e, meta 0x1a45972), peers [1,2,3,4] op hist []) 2026-02-11T23:30:58.070+0000 7fac4d160640 20 osd.0 107 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:30:58.070+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 50.906410217s delta_ub 1.980921268s) -- 0x3da3500 con 0x396b400 2026-02-11T23:30:58.070+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 50.906410217s delta_ub 1.980921268s) -- 0x3b1b500 con 0x3752400 2026-02-11T23:30:58.070+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 50.906410217s delta_ub 4.649119377s) -- 0x3b1a900 con 0x396b800 2026-02-11T23:30:58.070+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 50.906410217s delta_ub 4.649119377s) -- 0x3b1be00 con 0x34c1000 2026-02-11T23:30:58.070+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 50.906410217s delta_ub 7.310451508s) -- 0x3b1b380 con 0x3740000 2026-02-11T23:30:58.070+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 50.906410217s delta_ub 7.310451508s) -- 0x3b1b200 con 0x3740400 2026-02-11T23:30:58.070+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 50.906410217s delta_ub 9.886960983s) -- 0x3b1ad80 con 0x3740800 2026-02-11T23:30:58.070+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 50.906410217s delta_ub 9.886960983s) -- 0x3b1a000 con 0x3740c00 2026-02-11T23:30:58.070+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 18 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 48.925617218s delta_ub -1.980793238s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a000 con 0x396b400 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.070+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980898023s,-1.980793238s]) 2026-02-11T23:30:58.070+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 18 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 48.925636292s delta_ub -1.980777979s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x3752400 2026-02-11T23:30:58.070+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:30:58.070+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.070+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.070+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:58.070+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980900526s,-1.980777979s]) 2026-02-11T23:30:58.070+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 17 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 46.257484436s delta_ub -4.648928642s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ad80 con 0x396b800 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.070+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:58.070+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 15 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 43.596134186s delta_ub -7.310279369s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a000 con 0x3740000 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649069786s,-4.648928642s]) 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.070+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310428619s,-7.310279369s]) 2026-02-11T23:30:58.070+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 15 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 43.596153259s delta_ub -7.310258865s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ad80 con 0x3740400 2026-02-11T23:30:58.070+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 17 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 46.257503510s delta_ub -4.648907185s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x34c1000 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.070+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310415745s,-7.310258865s]) 2026-02-11T23:30:58.070+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 15 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 41.019622803s delta_ub -9.886791229s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a000 con 0x3740c00 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.070+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:58.070+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886953354s,-9.886791229s]) 2026-02-11T23:30:58.070+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:30:58.070+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.070+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.070+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:58.070+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649075508s,-4.648907185s]) 2026-02-11T23:30:58.070+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 15 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:30:58.071995+0000/50.906410217s send_stamp 41.019641876s delta_ub -9.886768341s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ad80 con 0x3740800 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.070+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:58.070+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886947632s,-9.886768341s]) 2026-02-11T23:30:58.109+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:30:58.109+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:58.109+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:58.109+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:58.109+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:58.109+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.102 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:58.109+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:58.110, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:58.109+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:58.109+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:58.109+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:58.109+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:58.109+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:58.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:58.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:58.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:28.170173+0000) 2026-02-11T23:30:58.757+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 15 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:30:58.758436+0000/41.705944061s send_stamp 41.705944061s delta_ub -9.886768341s) ==== 2033+0+0 (crc 0 0 0) 0x3b1a000 con 0x3f57400 2026-02-11T23:30:58.757+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 15 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:30:58.758436+0000/41.705944061s send_stamp 41.705944061s delta_ub -9.886768341s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x3f57800 2026-02-11T23:30:58.757+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886989594s,-9.886768341s]) 2026-02-11T23:30:58.757+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:58.758436+0000/41.705944061s send_stamp 51.592933655s delta_ub 9.886989594s) -- 0x3b1b380 con 0x3f57400 2026-02-11T23:30:58.757+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:58.757+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.757+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.757+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:58.757+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887006760s,-9.886768341s]) 2026-02-11T23:30:58.757+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:58.758436+0000/41.705944061s send_stamp 51.592948914s delta_ub 9.887006760s) -- 0x3b1b500 con 0x3f57800 2026-02-11T23:30:58.757+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:58.757+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:58.757+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:58.757+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:59.057+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:30:59.057+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:30:59.151+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:30:59.151+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:30:59.151+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:30:59.151+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:30:59.151+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:30:59.151+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.102 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:30:59.151+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:30:59.152, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:30:59.151+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:30:59.151+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:30:59.151+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:30:59.151+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:30:59.151+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:30:59.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:30:59.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:30:59.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:29.170244+0000) 2026-02-11T23:30:59.250+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 15 ==== osd_ping(ping e107 up_from 9 ping_stamp 2026-02-11T23:30:59.251488+0000/50.105052948s send_stamp 50.105052948s delta_ub -1.980777979s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b500 con 0x396a400 2026-02-11T23:30:59.250+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 15 ==== osd_ping(ping e107 up_from 9 ping_stamp 2026-02-11T23:30:59.251488+0000/50.105052948s send_stamp 50.105052948s delta_ub -1.980777979s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ad80 con 0x396a000 2026-02-11T23:30:59.250+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980916977s,-1.980777979s]) 2026-02-11T23:30:59.250+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:59.251488+0000/50.105052948s send_stamp 52.085968018s delta_ub 1.980916977s) -- 0x3b1be00 con 0x396a400 2026-02-11T23:30:59.250+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:30:59.250+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:59.250+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:59.250+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:59.250+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980941534s,-1.980777979s]) 2026-02-11T23:30:59.250+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:59.251488+0000/50.105052948s send_stamp 52.085994720s delta_ub 1.980941534s) -- 0x3b1b200 con 0x396a000 2026-02-11T23:30:59.250+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:30:59.250+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:59.250+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:59.250+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:59.480+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 12 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:30:59.481411+0000/45.005447388s send_stamp 45.005447388s delta_ub -7.310258865s) ==== 2033+0+0 (crc 0 0 0) 0x3b1be00 con 0x3f56800 2026-02-11T23:30:59.480+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 12 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:30:59.481411+0000/45.005447388s send_stamp 45.005447388s delta_ub -7.310258865s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b200 con 0x3f56c00 2026-02-11T23:30:59.480+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310470581s,-7.310258865s]) 2026-02-11T23:30:59.480+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:59.481411+0000/45.005447388s send_stamp 52.315917969s delta_ub 7.310470581s) -- 0x3b1b500 con 0x3f56800 2026-02-11T23:30:59.480+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:30:59.480+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:59.480+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:59.480+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:59.480+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310492516s,-7.310258865s]) 2026-02-11T23:30:59.480+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:59.481411+0000/45.005447388s send_stamp 52.315937042s delta_ub 7.310492516s) -- 0x3b1ad80 con 0x3f56c00 2026-02-11T23:30:59.480+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:30:59.480+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:59.480+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:59.480+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:59.857+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 16 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:30:59.858605+0000/42.806110382s send_stamp 42.806110382s delta_ub -9.886855125s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x3f57400 2026-02-11T23:30:59.857+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 16 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:30:59.858605+0000/42.806110382s send_stamp 42.806110382s delta_ub -9.886855125s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b500 con 0x3f57800 2026-02-11T23:30:59.857+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886982918s,-9.886855125s]) 2026-02-11T23:30:59.857+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:59.858605+0000/42.806110382s send_stamp 52.693096161s delta_ub 9.886982918s) -- 0x3b1a000 con 0x3f57400 2026-02-11T23:30:59.857+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:59.857+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:59.857+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:59.857+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:30:59.857+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887003899s,-9.886855125s]) 2026-02-11T23:30:59.857+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:30:59.858605+0000/42.806110382s send_stamp 52.693115234s delta_ub 9.887003899s) -- 0x3b1be00 con 0x3f57800 2026-02-11T23:30:59.857+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:30:59.857+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:30:59.857+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:30:59.857+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:00.067+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:00.067+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:00.122+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:00.122+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:00.122+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:00.122+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.097 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:00.122+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:00.123, recovery is active?:false restrictions:<....> 2026-02-11T23:31:00.122+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:00.122+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:00.122+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:00.122+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:00.122+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:00.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:00.168+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:00.168+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:30.170319+0000) 2026-02-11T23:31:00.168+0000 7fac616e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:00.970+0000 7fac4d160640 5 osd.0 107 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7ca37/0x118000, compress 0x0/0x0/0x0, omap 0xa68e, meta 0x1a45972), peers [1,2,3,4] op hist []) 2026-02-11T23:31:00.970+0000 7fac4d160640 20 osd.0 107 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:00.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 53.806613922s delta_ub 1.980941534s) -- 0x3b1af00 con 0x396b400 2026-02-11T23:31:00.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 53.806613922s delta_ub 1.980941534s) -- 0x3b1ac00 con 0x3752400 2026-02-11T23:31:00.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 53.806613922s delta_ub 4.649075508s) -- 0x3b1a480 con 0x396b800 2026-02-11T23:31:00.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 53.806613922s delta_ub 4.649075508s) -- 0x3b1b800 con 0x34c1000 2026-02-11T23:31:00.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 53.806613922s delta_ub 7.310492516s) -- 0x3b1bb00 con 0x3740000 2026-02-11T23:31:00.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 53.806613922s delta_ub 7.310492516s) -- 0x3919980 con 0x3740400 2026-02-11T23:31:00.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 53.806613922s delta_ub 9.887003899s) -- 0x3919200 con 0x3740800 2026-02-11T23:31:00.970+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 53.806613922s delta_ub 9.887003899s) -- 0x3da3080 con 0x3740c00 2026-02-11T23:31:00.970+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 19 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 51.825828552s delta_ub -1.980783820s) ==== 2033+0+0 (crc 0 0 0) 0x3da3080 con 0x396b400 2026-02-11T23:31:00.970+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 19 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 51.825843811s delta_ub -1.980769038s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b800 con 0x3752400 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:00.970+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980908751s,-1.980783820s]) 2026-02-11T23:31:00.970+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:00.970+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:00.970+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:00.970+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:00.970+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980905652s,-1.980769038s]) 2026-02-11T23:31:00.970+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 16 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 43.919818878s delta_ub -9.886793137s) ==== 2033+0+0 (crc 0 0 0) 0x3da3080 con 0x3740c00 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:00.970+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886940956s,-9.886793137s]) 2026-02-11T23:31:00.970+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 18 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 49.157695770s delta_ub -4.648915291s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b800 con 0x34c1000 2026-02-11T23:31:00.970+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:00.970+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:00.970+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:00.970+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:00.970+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649075508s,-4.648915291s]) 2026-02-11T23:31:00.970+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 16 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 46.496349335s delta_ub -7.310262680s) ==== 2033+0+0 (crc 0 0 0) 0x3919200 con 0x3740400 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:00.970+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310427666s,-7.310262680s]) 2026-02-11T23:31:00.970+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 18 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 49.157722473s delta_ub -4.648889065s) ==== 2033+0+0 (crc 0 0 0) 0x3919200 con 0x396b800 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:00.970+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649069786s,-4.648889065s]) 2026-02-11T23:31:00.970+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 16 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 46.496376038s delta_ub -7.310235500s) ==== 2033+0+0 (crc 0 0 0) 0x3da3080 con 0x3740000 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:00.970+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:00.970+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310423374s,-7.310235500s]) 2026-02-11T23:31:00.970+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 16 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:31:00.972196+0000/53.806613922s send_stamp 43.919826508s delta_ub -9.886783600s) ==== 2033+0+0 (crc 0 0 0) 0x3919200 con 0x3740800 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:00.970+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:00.970+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886975288s,-9.886783600s]) 2026-02-11T23:31:01.027+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:01.027+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:01.084+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:01.084+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:01.084+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:01.084+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:01.084+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:01.084+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.097 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:01.084+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:01.085, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:01.084+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:01.084+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:01.084+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:01.084+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:01.084+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:01.168+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:01.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:01.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:31.170409+0000) 2026-02-11T23:31:01.180+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 13 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:01.181603+0000/46.705638885s send_stamp 46.705638885s delta_ub -7.310235500s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b800 con 0x3f56800 2026-02-11T23:31:01.180+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310460567s,-7.310235500s]) 2026-02-11T23:31:01.180+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:01.181603+0000/46.705638885s send_stamp 54.016098022s delta_ub 7.310460567s) -- 0x3b1ac00 con 0x3f56800 2026-02-11T23:31:01.180+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:01.180+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:01.180+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:01.180+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:01.180+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 13 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:01.181603+0000/46.705638885s send_stamp 46.705638885s delta_ub -7.310235500s) ==== 2033+0+0 (crc 0 0 0) 0x3919200 con 0x3f56c00 2026-02-11T23:31:01.180+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310487270s,-7.310235500s]) 2026-02-11T23:31:01.180+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:01.181603+0000/46.705638885s send_stamp 54.016128540s delta_ub 7.310487270s) -- 0x3919980 con 0x3f56c00 2026-02-11T23:31:01.180+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:01.180+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:01.180+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:01.180+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:01.992+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:01.992+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:02.067+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:02.067+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:02.067+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:02.067+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:02.067+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:02.067+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.097 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:02.067+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:02.069, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:02.067+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:02.067+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:02.067+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:02.067+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:02.067+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:02.169+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:02.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:02.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:32.170487+0000) 2026-02-11T23:31:02.999+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:02.999+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:03.011+0000 7fac60ee5640 20 osd.0 107 reports for 0 queries 2026-02-11T23:31:03.011+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x34c3500 con 0x396a800 2026-02-11T23:31:03.011+0000 7fac60ee5640 15 osd.0 107 collect_pg_stats 2026-02-11T23:31:03.011+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836496 v 0) -- 0x3df0900 con 0x396a800 2026-02-11T23:31:03.078+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:03.078+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:03.078+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:03.078+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:03.078+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:03.079+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.097 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:03.079+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:03.080, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:03.079+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:03.079+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:03.079+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:03.079+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:03.079+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:03.169+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:03.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:03.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:33.170564+0000) 2026-02-11T23:31:03.318+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 13 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:31:03.319671+0000/51.505065918s send_stamp 51.505065918s delta_ub -4.648889065s) ==== 2033+0+0 (crc 0 0 0) 0x3da3080 con 0x3f56400 2026-02-11T23:31:03.318+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649141312s,-4.648889065s]) 2026-02-11T23:31:03.318+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:03.319671+0000/51.505065918s send_stamp 56.154209137s delta_ub 4.649141312s) -- 0x3b1bb00 con 0x3f56400 2026-02-11T23:31:03.318+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:03.318+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:03.318+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:03.318+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:03.318+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 13 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:31:03.319671+0000/51.505065918s send_stamp 51.505065918s delta_ub -4.648889065s) ==== 2033+0+0 (crc 0 0 0) 0x3919980 con 0x3f56000 2026-02-11T23:31:03.318+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649171352s,-4.648889065s]) 2026-02-11T23:31:03.318+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:03.319671+0000/51.505065918s send_stamp 56.154239655s delta_ub 4.649171352s) -- 0x3919200 con 0x3f56000 2026-02-11T23:31:03.318+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:03.318+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:03.318+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:03.318+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:04.010+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:04.010+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:04.080+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 14 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:04.081800+0000/49.605834961s send_stamp 49.605834961s delta_ub -7.310318947s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ac00 con 0x3f56800 2026-02-11T23:31:04.080+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310458183s,-7.310318947s]) 2026-02-11T23:31:04.080+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:04.081800+0000/49.605834961s send_stamp 56.916294098s delta_ub 7.310458183s) -- 0x3b1b800 con 0x3f56800 2026-02-11T23:31:04.080+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:04.080+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:04.080+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:04.080+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 14 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:04.081800+0000/49.605834961s send_stamp 49.605834961s delta_ub -7.310318947s) ==== 2033+0+0 (crc 0 0 0) 0x3919200 con 0x3f56c00 2026-02-11T23:31:04.080+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:04.080+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310484409s,-7.310318947s]) 2026-02-11T23:31:04.080+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:04.081800+0000/49.605834961s send_stamp 56.916320801s delta_ub 7.310484409s) -- 0x3919980 con 0x3f56c00 2026-02-11T23:31:04.080+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:04.080+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:04.080+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:04.080+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:04.088+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:04.088+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:04.088+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:04.088+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.097 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:04.088+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:04.090, recovery is active?:false restrictions:<....> 2026-02-11T23:31:04.088+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:04.088+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:04.088+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:04.088+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:04.088+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:04.169+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:04.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:04.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:34.170639+0000) 2026-02-11T23:31:04.550+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 16 ==== osd_ping(ping e107 up_from 9 ping_stamp 2026-02-11T23:31:04.551660+0000/55.405223846s send_stamp 55.405223846s delta_ub -1.980769038s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b800 con 0x396a400 2026-02-11T23:31:04.550+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980925441s,-1.980769038s]) 2026-02-11T23:31:04.550+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:04.551660+0000/55.405223846s send_stamp 57.386146545s delta_ub 1.980925441s) -- 0x3b1ac00 con 0x396a400 2026-02-11T23:31:04.550+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:04.550+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:04.550+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:04.550+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:04.550+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 16 ==== osd_ping(ping e107 up_from 9 ping_stamp 2026-02-11T23:31:04.551660+0000/55.405223846s send_stamp 55.405223846s delta_ub -1.980769038s) ==== 2033+0+0 (crc 0 0 0) 0x3919980 con 0x396a000 2026-02-11T23:31:04.550+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980953097s,-1.980769038s]) 2026-02-11T23:31:04.550+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:04.551660+0000/55.405223846s send_stamp 57.386177063s delta_ub 1.980953097s) -- 0x3919200 con 0x396a000 2026-02-11T23:31:04.550+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:04.550+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:04.550+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:04.550+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:05.010+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:05.010+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:05.115+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:05.116+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:05.116+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:05.116+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:05.116+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:05.116+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.093 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:05.116+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:05.117, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:05.116+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:05.116+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:05.116+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:05.116+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:05.116+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:05.157+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 17 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:31:05.158765+0000/48.106269836s send_stamp 48.106269836s delta_ub -9.886783600s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bb00 con 0x3f57400 2026-02-11T23:31:05.157+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 17 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:31:05.158765+0000/48.106269836s send_stamp 48.106269836s delta_ub -9.886783600s) ==== 2033+0+0 (crc 0 0 0) 0x3b1ac00 con 0x3f57800 2026-02-11T23:31:05.157+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886976242s,-9.886783600s]) 2026-02-11T23:31:05.157+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:05.158765+0000/48.106269836s send_stamp 57.993247986s delta_ub 9.886976242s) -- 0x3da3080 con 0x3f57400 2026-02-11T23:31:05.157+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:05.157+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:05.157+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:05.157+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:05.157+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886997223s,-9.886783600s]) 2026-02-11T23:31:05.157+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:05.158765+0000/48.106269836s send_stamp 57.993270874s delta_ub 9.886997223s) -- 0x3b1b800 con 0x3f57800 2026-02-11T23:31:05.157+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:05.157+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:05.157+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:05.157+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:05.169+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:05.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:05.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:35.170714+0000) 2026-02-11T23:31:05.986+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:05.986+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:06.122+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:06.122+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:06.122+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:06.122+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:06.122+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:06.122+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.093 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:06.122+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:06.124, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:06.122+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:06.122+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:06.122+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:06.122+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:06.122+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:06.169+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:06.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:06.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:36.170788+0000) 2026-02-11T23:31:06.270+0000 7fac4d160640 5 osd.0 107 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7ca37/0x118000, compress 0x0/0x0/0x0, omap 0xa68e, meta 0x1a45972), peers [1,2,3,4] op hist []) 2026-02-11T23:31:06.270+0000 7fac4d160640 20 osd.0 107 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:06.270+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 59.106807709s delta_ub 1.980953097s) -- 0x3da3800 con 0x396b400 2026-02-11T23:31:06.271+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 59.106807709s delta_ub 1.980953097s) -- 0x3da2c00 con 0x3752400 2026-02-11T23:31:06.271+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 59.106807709s delta_ub 4.649171352s) -- 0x3da2480 con 0x396b800 2026-02-11T23:31:06.271+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 59.106807709s delta_ub 4.649171352s) -- 0x3da2d80 con 0x34c1000 2026-02-11T23:31:06.271+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 59.106807709s delta_ub 7.310484409s) -- 0x3e02000 con 0x3740000 2026-02-11T23:31:06.271+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 59.106807709s delta_ub 7.310484409s) -- 0x3e02180 con 0x3740400 2026-02-11T23:31:06.271+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 59.106807709s delta_ub 9.886997223s) -- 0x3e02300 con 0x3740800 2026-02-11T23:31:06.271+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 59.106807709s delta_ub 9.886997223s) -- 0x3e02480 con 0x3740c00 2026-02-11T23:31:06.271+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 20 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 57.126029968s delta_ub -1.980775833s) ==== 2033+0+0 (crc 0 0 0) 0x3da2d80 con 0x3752400 2026-02-11T23:31:06.271+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 20 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 57.126049042s delta_ub -1.980761290s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x396b400 2026-02-11T23:31:06.271+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:06.271+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:06.271+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:06.271+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:06.271+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980910301s,-1.980775833s]) 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:06.271+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980910182s,-1.980761290s]) 2026-02-11T23:31:06.271+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 19 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 54.457893372s delta_ub -4.648914337s) ==== 2033+0+0 (crc 0 0 0) 0x3da2d80 con 0x34c1000 2026-02-11T23:31:06.271+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:06.271+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:06.271+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:06.271+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:06.271+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649073601s,-4.648914337s]) 2026-02-11T23:31:06.271+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 17 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 49.220005035s delta_ub -9.886801720s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x3740c00 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:06.271+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886970520s,-9.886801720s]) 2026-02-11T23:31:06.271+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 17 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 51.796550751s delta_ub -7.310256004s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x3740000 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:06.271+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 17 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 49.220035553s delta_ub -9.886772156s) ==== 2033+0+0 (crc 0 0 0) 0x3e02300 con 0x3740800 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:06.271+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:06.271+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310439587s,-7.310256004s]) 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:06.271+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886962891s,-9.886772156s]) 2026-02-11T23:31:06.271+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 19 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 54.457920074s delta_ub -4.648884296s) ==== 2033+0+0 (crc 0 0 0) 0x3e02300 con 0x396b800 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:06.271+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649096489s,-4.648884296s]) 2026-02-11T23:31:06.271+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 17 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:31:06.272391+0000/59.106807709s send_stamp 51.796573639s delta_ub -7.310234070s) ==== 2033+0+0 (crc 0 0 0) 0x3e02300 con 0x3740400 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:06.271+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:06.271+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310456276s,-7.310234070s]) 2026-02-11T23:31:06.818+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 14 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:31:06.819891+0000/55.005287170s send_stamp 55.005287170s delta_ub -4.648884296s) ==== 2033+0+0 (crc 0 0 0) 0x3e02300 con 0x3f56000 2026-02-11T23:31:06.818+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 14 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:31:06.819891+0000/55.005287170s send_stamp 55.005287170s delta_ub -4.648884296s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x3f56400 2026-02-11T23:31:06.818+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649125576s,-4.648884296s]) 2026-02-11T23:31:06.818+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:06.819891+0000/55.005287170s send_stamp 59.654415131s delta_ub 4.649125576s) -- 0x3e02180 con 0x3f56000 2026-02-11T23:31:06.818+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:06.818+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:06.818+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:06.818+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:06.818+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649153233s,-4.648884296s]) 2026-02-11T23:31:06.818+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:06.819891+0000/55.005287170s send_stamp 59.654438019s delta_ub 4.649153233s) -- 0x3e02000 con 0x3f56400 2026-02-11T23:31:06.818+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:06.818+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:06.818+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:06.818+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:07.001+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:07.001+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:07.154+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:07.154+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:07.154+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:07.154+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.093 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:07.154+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:07.156, recovery is active?:false restrictions:<....> 2026-02-11T23:31:07.154+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:07.154+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:07.154+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:07.154+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:07.154+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:07.169+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:07.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:07.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:37.170859+0000) 2026-02-11T23:31:07.580+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 15 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:07.581990+0000/53.106029510s send_stamp 53.106029510s delta_ub -7.310234070s) ==== 2033+0+0 (crc 0 0 0) 0x3e02180 con 0x3f56c00 2026-02-11T23:31:07.580+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 15 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:07.581990+0000/53.106029510s send_stamp 53.106029510s delta_ub -7.310234070s) ==== 2033+0+0 (crc 0 0 0) 0x3da2d80 con 0x3f56800 2026-02-11T23:31:07.580+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310473919s,-7.310234070s]) 2026-02-11T23:31:07.580+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:07.581990+0000/53.106029510s send_stamp 60.416500092s delta_ub 7.310473919s) -- 0x3e02300 con 0x3f56c00 2026-02-11T23:31:07.580+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:07.580+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:07.580+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:07.580+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:07.580+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310495377s,-7.310234070s]) 2026-02-11T23:31:07.580+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:07.581990+0000/53.106029510s send_stamp 60.416519165s delta_ub 7.310495377s) -- 0x3da2c00 con 0x3f56800 2026-02-11T23:31:07.580+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:07.580+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:07.580+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:07.580+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:08.012+0000 7fac60ee5640 20 osd.0 107 reports for 0 queries 2026-02-11T23:31:08.012+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x34c3c00 con 0x396a800 2026-02-11T23:31:08.012+0000 7fac60ee5640 15 osd.0 107 collect_pg_stats 2026-02-11T23:31:08.012+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836497 v 0) -- 0x3df0c00 con 0x396a800 2026-02-11T23:31:08.042+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:08.042+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:08.138+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:08.138+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:08.138+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:08.138+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:08.138+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:08.138+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.093 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:08.138+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:08.140, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:08.138+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:08.138+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:08.138+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:08.138+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:08.138+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:08.169+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:08.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:08.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:38.170931+0000) 2026-02-11T23:31:09.040+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:09.040+0000 7fac68beb640 10 osd.0 107 maybe_update_heartbeat_peers forcing update after 20.905157 seconds 2026-02-11T23:31:09.040+0000 7fac68beb640 10 osd.0 107 reset_heartbeat_peers 2026-02-11T23:31:09.040+0000 7fac68beb640 10 osd.0 107 maybe_update_heartbeat_peers updating 2026-02-11T23:31:09.040+0000 7fac68beb640 10 osd.0 107 adding neighbor peer osd.1 2026-02-11T23:31:09.040+0000 7fac68beb640 10 osd.0 107 adding neighbor peer osd.4 2026-02-11T23:31:09.040+0000 7fac68beb640 10 osd.0 107 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:31:09.040+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:09.169+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:09.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:09.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:39.171004+0000) 2026-02-11T23:31:09.171+0000 7fac4d160640 5 osd.0 107 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7ca37/0x118000, compress 0x0/0x0/0x0, omap 0xa68e, meta 0x1a45972), peers [1,2,3,4] op hist []) 2026-02-11T23:31:09.171+0000 7fac4d160640 20 osd.0 107 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:09.171+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 62.006996155s delta_ub 1.980910182s) -- 0x3e02600 con 0x396b400 2026-02-11T23:31:09.171+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 62.006996155s delta_ub 1.980910182s) -- 0x3e02780 con 0x3752400 2026-02-11T23:31:09.171+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 62.006996155s delta_ub 4.649153233s) -- 0x3e02900 con 0x396b800 2026-02-11T23:31:09.171+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 62.006996155s delta_ub 4.649153233s) -- 0x3e02a80 con 0x34c1000 2026-02-11T23:31:09.171+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 62.006996155s delta_ub 7.310495377s) -- 0x3e02c00 con 0x3740000 2026-02-11T23:31:09.171+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 62.006996155s delta_ub 7.310495377s) -- 0x3e02d80 con 0x3740400 2026-02-11T23:31:09.171+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 62.006996155s delta_ub 9.886962891s) -- 0x3e02f00 con 0x3740800 2026-02-11T23:31:09.171+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 62.006996155s delta_ub 9.886962891s) -- 0x3e03080 con 0x3740c00 2026-02-11T23:31:09.171+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 21 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 60.026229858s delta_ub -1.980768204s) ==== 2033+0+0 (crc 0 0 0) 0x3e03080 con 0x396b400 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.171+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 21 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 60.026248932s delta_ub -1.980748892s) ==== 2033+0+0 (crc 0 0 0) 0x3e02a80 con 0x3752400 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.171+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980910182s,-1.980768204s]) 2026-02-11T23:31:09.171+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:09.171+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 20 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 57.358089447s delta_ub -4.648907185s) ==== 2033+0+0 (crc 0 0 0) 0x3e02f00 con 0x396b800 2026-02-11T23:31:09.171+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 18 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 52.120193481s delta_ub -9.886799812s) ==== 2033+0+0 (crc 0 0 0) 0x3e03080 con 0x3740c00 2026-02-11T23:31:09.171+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.171+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.171+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.171+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980904698s,-1.980748892s]) 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.171+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649084568s,-4.648907185s]) 2026-02-11T23:31:09.171+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 20 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 57.358112335s delta_ub -4.648884296s) ==== 2033+0+0 (crc 0 0 0) 0x3e02a80 con 0x34c1000 2026-02-11T23:31:09.171+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:09.171+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.171+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.171+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.171+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649070740s,-4.648884296s]) 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.171+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 18 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 52.120216370s delta_ub -9.886779785s) ==== 2033+0+0 (crc 0 0 0) 0x3e02f00 con 0x3740800 2026-02-11T23:31:09.171+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886995316s,-9.886799812s]) 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.171+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 18 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 54.696727753s delta_ub -7.310269356s) ==== 2033+0+0 (crc 0 0 0) 0x3e03080 con 0x3740000 2026-02-11T23:31:09.171+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886983871s,-9.886779785s]) 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.171+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.171+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310487747s,-7.310269356s]) 2026-02-11T23:31:09.171+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 18 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:31:09.172580+0000/62.006996155s send_stamp 54.696746826s delta_ub -7.310247421s) ==== 2033+0+0 (crc 0 0 0) 0x3e02f00 con 0x3740400 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.171+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.171+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310472012s,-7.310247421s]) 2026-02-11T23:31:09.173+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:09.173+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:09.173+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:09.173+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:09.173+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:09.173+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.093 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:09.173+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:09.174, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:09.173+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:09.173+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:09.173+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:09.173+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:09.173+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:09.250+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 17 ==== osd_ping(ping e107 up_from 9 ping_stamp 2026-02-11T23:31:09.251851+0000/60.105415344s send_stamp 60.105415344s delta_ub -1.980748892s) ==== 2033+0+0 (crc 0 0 0) 0x3e02a80 con 0x396a400 2026-02-11T23:31:09.250+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980905175s,-1.980748892s]) 2026-02-11T23:31:09.250+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:09.251851+0000/60.105415344s send_stamp 62.086318970s delta_ub 1.980905175s) -- 0x3e02780 con 0x396a400 2026-02-11T23:31:09.250+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:09.250+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.250+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.250+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.250+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 17 ==== osd_ping(ping e107 up_from 9 ping_stamp 2026-02-11T23:31:09.251851+0000/60.105415344s send_stamp 60.105415344s delta_ub -1.980748892s) ==== 2033+0+0 (crc 0 0 0) 0x3e02f00 con 0x396a000 2026-02-11T23:31:09.250+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980935931s,-1.980748892s]) 2026-02-11T23:31:09.250+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:09.251851+0000/60.105415344s send_stamp 62.086353302s delta_ub 1.980935931s) -- 0x3e02d80 con 0x396a000 2026-02-11T23:31:09.250+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:09.250+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.250+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.250+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.257+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 18 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:31:09.258924+0000/52.206432343s send_stamp 52.206432343s delta_ub -9.886779785s) ==== 2033+0+0 (crc 0 0 0) 0x3e02780 con 0x3f57800 2026-02-11T23:31:09.257+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 18 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:31:09.258924+0000/52.206432343s send_stamp 52.206432343s delta_ub -9.886779785s) ==== 2033+0+0 (crc 0 0 0) 0x3e03080 con 0x3f57400 2026-02-11T23:31:09.257+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886955261s,-9.886779785s]) 2026-02-11T23:31:09.257+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:09.258924+0000/52.206432343s send_stamp 62.093387604s delta_ub 9.886955261s) -- 0x3e02a80 con 0x3f57800 2026-02-11T23:31:09.257+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:09.257+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.257+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.257+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.257+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886969566s,-9.886779785s]) 2026-02-11T23:31:09.257+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:09.258924+0000/52.206432343s send_stamp 62.093402863s delta_ub 9.886969566s) -- 0x3e02480 con 0x3f57400 2026-02-11T23:31:09.257+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:09.257+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.257+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.257+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.280+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 16 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:09.282152+0000/54.806186676s send_stamp 54.806186676s delta_ub -7.310247421s) ==== 2033+0+0 (crc 0 0 0) 0x3e02a80 con 0x3f56800 2026-02-11T23:31:09.280+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310434341s,-7.310247421s]) 2026-02-11T23:31:09.280+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:09.282152+0000/54.806186676s send_stamp 62.116622925s delta_ub 7.310434341s) -- 0x3e02780 con 0x3f56800 2026-02-11T23:31:09.280+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:09.280+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.280+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.280+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.280+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 16 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:09.282152+0000/54.806186676s send_stamp 54.806186676s delta_ub -7.310247421s) ==== 2033+0+0 (crc 0 0 0) 0x3e02d80 con 0x3f56c00 2026-02-11T23:31:09.280+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310452938s,-7.310247421s]) 2026-02-11T23:31:09.280+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:09.282152+0000/54.806186676s send_stamp 62.116641998s delta_ub 7.310452938s) -- 0x3e02f00 con 0x3f56c00 2026-02-11T23:31:09.280+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:09.280+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.280+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.280+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.718+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 15 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:31:09.720092+0000/57.905487061s send_stamp 57.905487061s delta_ub -4.648884296s) ==== 2033+0+0 (crc 0 0 0) 0x3e02f00 con 0x3f56000 2026-02-11T23:31:09.718+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 15 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:31:09.720092+0000/57.905487061s send_stamp 57.905487061s delta_ub -4.648884296s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x3f56400 2026-02-11T23:31:09.718+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649121761s,-4.648884296s]) 2026-02-11T23:31:09.718+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:09.720092+0000/57.905487061s send_stamp 62.554607391s delta_ub 4.649121761s) -- 0x3e02d80 con 0x3f56000 2026-02-11T23:31:09.718+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:09.718+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.718+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.718+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.718+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649144173s,-4.648884296s]) 2026-02-11T23:31:09.718+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:09.720092+0000/57.905487061s send_stamp 62.554634094s delta_ub 4.649144173s) -- 0x3e03080 con 0x3f56400 2026-02-11T23:31:09.718+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:09.718+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.718+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.718+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.780+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 17 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:09.782344+0000/55.306377411s send_stamp 55.306377411s delta_ub -7.310337543s) ==== 2033+0+0 (crc 0 0 0) 0x3e02780 con 0x3f56800 2026-02-11T23:31:09.781+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310451031s,-7.310337543s]) 2026-02-11T23:31:09.781+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:09.782344+0000/55.306377411s send_stamp 62.616828918s delta_ub 7.310451031s) -- 0x3e02a80 con 0x3f56800 2026-02-11T23:31:09.781+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:09.781+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.781+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.781+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:09.781+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 17 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:09.782344+0000/55.306377411s send_stamp 55.306377411s delta_ub -7.310337543s) ==== 2033+0+0 (crc 0 0 0) 0x3e02d80 con 0x3f56c00 2026-02-11T23:31:09.781+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310466766s,-7.310337543s]) 2026-02-11T23:31:09.781+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:09.782344+0000/55.306377411s send_stamp 62.616847992s delta_ub 7.310466766s) -- 0x3e02f00 con 0x3f56c00 2026-02-11T23:31:09.781+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:09.781+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:09.781+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:09.781+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:10.083+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:10.083+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:10.127+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:10.127+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:10.127+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:10.127+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:10.127+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:10.127+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.088 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:10.127+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:10.128, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:10.127+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:10.127+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:10.127+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:10.127+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:10.127+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:10.169+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:10.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:10.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:40.171074+0000) 2026-02-11T23:31:10.169+0000 7fac616e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:11.082+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:11.082+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:11.169+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:11.169+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:11.169+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:41.171169+0000) 2026-02-11T23:31:11.175+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:11.175+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:11.175+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:11.175+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:11.175+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:11.175+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.088 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:11.175+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:11.176, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:11.175+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:11.175+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:11.175+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:11.175+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:11.175+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:12.077+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:12.077+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:12.158+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 19 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:31:12.159075+0000/55.106578827s send_stamp 55.106578827s delta_ub -9.886831284s) ==== 2033+0+0 (crc 0 0 0) 0x3e03080 con 0x3f57400 2026-02-11T23:31:12.158+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886977196s,-9.886831284s]) 2026-02-11T23:31:12.158+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:12.159075+0000/55.106578827s send_stamp 64.993560791s delta_ub 9.886977196s) -- 0x3e02480 con 0x3f57400 2026-02-11T23:31:12.158+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 19 ==== osd_ping(ping e107 up_from 21 ping_stamp 2026-02-11T23:31:12.159075+0000/55.106578827s send_stamp 55.106578827s delta_ub -9.886831284s) ==== 2033+0+0 (crc 0 0 0) 0x3e02a80 con 0x3f57800 2026-02-11T23:31:12.158+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:12.158+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:12.158+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:12.158+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:12.158+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887001991s,-9.886831284s]) 2026-02-11T23:31:12.158+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:12.159075+0000/55.106578827s send_stamp 64.993583679s delta_ub 9.887001991s) -- 0x3e02780 con 0x3f57800 2026-02-11T23:31:12.158+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:12.158+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:12.158+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:12.158+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:12.169+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:12.169+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:12.169+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:12.169+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:12.169+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:12.169+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.088 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:12.169+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:12.169, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:12.169+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:12.169+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:12.169+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:12.169+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:12.169+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:12.170+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:12.170+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:12.170+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:42.171242+0000) 2026-02-11T23:31:13.013+0000 7fac60ee5640 20 osd.0 107 reports for 0 queries 2026-02-11T23:31:13.013+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x34c3180 con 0x396a800 2026-02-11T23:31:13.013+0000 7fac60ee5640 15 osd.0 107 collect_pg_stats 2026-02-11T23:31:13.013+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836498 v 0) -- 0x3df0f00 con 0x396a800 2026-02-11T23:31:13.085+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:13.085+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:13.170+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:13.170+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:13.170+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:43.171316+0000) 2026-02-11T23:31:13.184+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:13.184+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:13.184+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:13.185+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:13.185+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:13.185+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.088 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:13.185+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:13.185, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:13.185+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:13.185+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:13.185+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:13.185+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:13.185+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:13.819+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 16 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:31:13.820278+0000/62.005672455s send_stamp 62.005672455s delta_ub -4.648945332s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x3f56400 2026-02-11T23:31:13.819+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649107456s,-4.648945332s]) 2026-02-11T23:31:13.819+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:13.820278+0000/62.005672455s send_stamp 66.654785156s delta_ub 4.649107456s) -- 0x3e03080 con 0x3f56400 2026-02-11T23:31:13.819+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 16 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:31:13.820278+0000/62.005672455s send_stamp 62.005672455s delta_ub -4.648945332s) ==== 2033+0+0 (crc 0 0 0) 0x3e02f00 con 0x3f56000 2026-02-11T23:31:13.819+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:13.819+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:13.819+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:13.819+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:13.819+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649130344s,-4.648945332s]) 2026-02-11T23:31:13.819+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:13.820278+0000/62.005672455s send_stamp 66.654800415s delta_ub 4.649130344s) -- 0x3e02d80 con 0x3f56000 2026-02-11T23:31:13.819+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:13.820+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:13.820+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:13.820+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:13.951+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 18 ==== osd_ping(ping e107 up_from 9 ping_stamp 2026-02-11T23:31:13.952048+0000/64.805610657s send_stamp 64.805610657s delta_ub -1.980810285s) ==== 2033+0+0 (crc 0 0 0) 0x3e02780 con 0x396a400 2026-02-11T23:31:13.951+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980933189s,-1.980810285s]) 2026-02-11T23:31:13.951+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 18 ==== osd_ping(ping e107 up_from 9 ping_stamp 2026-02-11T23:31:13.952048+0000/64.805610657s send_stamp 64.805610657s delta_ub -1.980810285s) ==== 2033+0+0 (crc 0 0 0) 0x3e02d80 con 0x396a000 2026-02-11T23:31:13.951+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:13.952048+0000/64.805610657s send_stamp 66.786544800s delta_ub 1.980933189s) -- 0x3e02a80 con 0x396a400 2026-02-11T23:31:13.951+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:13.951+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:13.951+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:13.951+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:13.951+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980953932s,-1.980810285s]) 2026-02-11T23:31:13.951+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:13.952048+0000/64.805610657s send_stamp 66.786567688s delta_ub 1.980953932s) -- 0x3e02f00 con 0x396a000 2026-02-11T23:31:13.951+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:13.951+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:13.951+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:13.951+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:14.054+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:14.054+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:14.156+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:14.156+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:14.156+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:14.156+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:14.156+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:14.156+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.088 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:14.156+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:14.157, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:14.156+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:14.156+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:14.156+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:14.156+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:14.156+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:14.170+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:14.170+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:14.170+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:44.171383+0000) 2026-02-11T23:31:15.052+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:15.052+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:15.072+0000 7fac4d160640 5 osd.0 107 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7ca37/0x118000, compress 0x0/0x0/0x0, omap 0xa68e, meta 0x1a45972), peers [1,2,3,4] op hist []) 2026-02-11T23:31:15.072+0000 7fac4d160640 20 osd.0 107 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:15.072+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 67.907180786s delta_ub 1.980953932s) -- 0x3e03200 con 0x396b400 2026-02-11T23:31:15.072+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 67.907180786s delta_ub 1.980953932s) -- 0x3e03380 con 0x3752400 2026-02-11T23:31:15.072+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 67.907180786s delta_ub 4.649130344s) -- 0x3e03500 con 0x396b800 2026-02-11T23:31:15.072+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 67.907180786s delta_ub 4.649130344s) -- 0x3e03680 con 0x34c1000 2026-02-11T23:31:15.072+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 67.907180786s delta_ub 7.310466766s) -- 0x3e03800 con 0x3740000 2026-02-11T23:31:15.072+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 67.907180786s delta_ub 7.310466766s) -- 0x3e03980 con 0x3740400 2026-02-11T23:31:15.072+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 67.907180786s delta_ub 9.887001991s) -- 0x3e03b00 con 0x3740800 2026-02-11T23:31:15.072+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 67.907180786s delta_ub 9.887001991s) -- 0x3e02c00 con 0x3740c00 2026-02-11T23:31:15.072+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 22 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 65.926391602s delta_ub -1.980784655s) ==== 2033+0+0 (crc 0 0 0) 0x3e02c00 con 0x396b400 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.072+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980914950s,-1.980784655s]) 2026-02-11T23:31:15.072+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 19 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 60.596900940s delta_ub -7.310277939s) ==== 2033+0+0 (crc 0 0 0) 0x3e02c00 con 0x3740000 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.072+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 21 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 63.258255005s delta_ub -4.648924351s) ==== 2033+0+0 (crc 0 0 0) 0x3e03b00 con 0x396b800 2026-02-11T23:31:15.072+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 22 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 65.926422119s delta_ub -1.980754375s) ==== 2033+0+0 (crc 0 0 0) 0x3e02780 con 0x3752400 2026-02-11T23:31:15.072+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310427666s,-7.310277939s]) 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.072+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 19 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 58.020362854s delta_ub -9.886812210s) ==== 2033+0+0 (crc 0 0 0) 0x3e02c00 con 0x3740c00 2026-02-11T23:31:15.072+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649083614s,-4.648924351s]) 2026-02-11T23:31:15.072+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 19 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 60.596923828s delta_ub -7.310254097s) ==== 2033+0+0 (crc 0 0 0) 0x3e03b00 con 0x3740400 2026-02-11T23:31:15.072+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:15.072+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.072+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.072+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:15.072+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980925322s,-1.980754375s]) 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.072+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:15.072+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887012482s,-9.886812210s]) 2026-02-11T23:31:15.072+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 21 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 63.258281708s delta_ub -4.648898602s) ==== 2033+0+0 (crc 0 0 0) 0x3e02780 con 0x34c1000 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.072+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310463428s,-7.310254097s]) 2026-02-11T23:31:15.072+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:15.072+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.072+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 19 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:31:15.072762+0000/67.907180786s send_stamp 58.020397186s delta_ub -9.886779785s) ==== 2033+0+0 (crc 0 0 0) 0x3e03b00 con 0x3740800 2026-02-11T23:31:15.072+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.072+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:15.072+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649119854s,-4.648898602s]) 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.072+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:15.072+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887018204s,-9.886779785s]) 2026-02-11T23:31:15.082+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 18 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:15.082454+0000/60.606491089s send_stamp 60.606491089s delta_ub -7.310254097s) ==== 2033+0+0 (crc 0 0 0) 0x3e02780 con 0x3f56800 2026-02-11T23:31:15.082+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310429573s,-7.310254097s]) 2026-02-11T23:31:15.082+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:15.082454+0000/60.606491089s send_stamp 67.916923523s delta_ub 7.310429573s) -- 0x3e02a80 con 0x3f56800 2026-02-11T23:31:15.082+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:15.082+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.082+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.082+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 18 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:15.082454+0000/60.606491089s send_stamp 60.606491089s delta_ub -7.310254097s) ==== 2033+0+0 (crc 0 0 0) 0x3e03b00 con 0x3f56c00 2026-02-11T23:31:15.082+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:15.082+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310443878s,-7.310254097s]) 2026-02-11T23:31:15.082+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:15.082454+0000/60.606491089s send_stamp 67.916938782s delta_ub 7.310443878s) -- 0x3e03980 con 0x3f56c00 2026-02-11T23:31:15.082+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:15.082+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.082+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.082+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:15.151+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:15.151+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:15.151+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:15.151+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.085 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:15.151+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:15.151, recovery is active?:false restrictions:<....> 2026-02-11T23:31:15.151+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:15.151+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:15.151+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:15.151+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:15.151+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:15.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:15.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:15.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:45.171420+0000) 2026-02-11T23:31:15.582+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 19 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:15.582623+0000/61.106658936s send_stamp 61.106658936s delta_ub -7.310304642s) ==== 2033+0+0 (crc 0 0 0) 0x3e03980 con 0x3f56c00 2026-02-11T23:31:15.582+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 19 ==== osd_ping(ping e107 up_from 17 ping_stamp 2026-02-11T23:31:15.582623+0000/61.106658936s send_stamp 61.106658936s delta_ub -7.310304642s) ==== 2033+0+0 (crc 0 0 0) 0x3e02a80 con 0x3f56800 2026-02-11T23:31:15.582+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310473919s,-7.310304642s]) 2026-02-11T23:31:15.582+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:15.582623+0000/61.106658936s send_stamp 68.417137146s delta_ub 7.310473919s) -- 0x3e03b00 con 0x3f56c00 2026-02-11T23:31:15.582+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:15.582+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.582+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.582+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:15.582+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310500145s,-7.310304642s]) 2026-02-11T23:31:15.582+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:15.582623+0000/61.106658936s send_stamp 68.417160034s delta_ub 7.310500145s) -- 0x3e02780 con 0x3f56800 2026-02-11T23:31:15.582+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:15.582+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:15.582+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:15.582+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:16.059+0000 7fac68beb640 10 osd.0 107 tick 2026-02-11T23:31:16.059+0000 7fac68beb640 20 osd.0 107 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:16.101+0000 7fac683ea640 10 osd.0 107 tick_without_osd_lock 2026-02-11T23:31:16.101+0000 7fac683ea640 0 osd.0 107 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:16.101+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:16.101+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:16.101+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:16.101+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.085 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:16.101+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:16.102, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:16.101+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:16.101+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:16.101+0000 7fac683ea640 20 osd.0 107 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:16.101+0000 7fac683ea640 10 osd.0 107 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:16.101+0000 7fac683ea640 20 osd.0 107 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:16.120+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 17 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:31:16.120470+0000/64.305862427s send_stamp 64.305862427s delta_ub -4.648898602s) ==== 2033+0+0 (crc 0 0 0) 0x3e03b00 con 0x3f56000 2026-02-11T23:31:16.120+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649102688s,-4.648898602s]) 2026-02-11T23:31:16.120+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 17 ==== osd_ping(ping e107 up_from 13 ping_stamp 2026-02-11T23:31:16.120470+0000/64.305862427s send_stamp 64.305862427s delta_ub -4.648898602s) ==== 2033+0+0 (crc 0 0 0) 0x3e02c00 con 0x3f56400 2026-02-11T23:31:16.120+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:16.120470+0000/64.305862427s send_stamp 68.954971313s delta_ub 4.649102688s) -- 0x3e03980 con 0x3f56000 2026-02-11T23:31:16.120+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:16.120+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:16.120+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:16.120+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:16.120+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649148464s,-4.648898602s]) 2026-02-11T23:31:16.120+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e107 up_from 5 ping_stamp 2026-02-11T23:31:16.120470+0000/64.305862427s send_stamp 68.955009460s delta_ub 4.649148464s) -- 0x3e03800 con 0x3f56400 2026-02-11T23:31:16.120+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:16.120+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:16.120+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:16.120+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:16.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:16.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:16.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:46.171493+0000) 2026-02-11T23:31:16.772+0000 7fac4d160640 5 osd.0 107 heartbeat osd_stat(store_statfs(0x384000/0x0/0x384000, data 0x7ca37/0x118000, compress 0x0/0x0/0x0, omap 0xa68e, meta 0x1a45972), peers [1,2,3,4] op hist []) 2026-02-11T23:31:16.772+0000 7fac4d160640 20 osd.0 107 check_full_status cur ratio 0, physical ratio 0, new state none 2026-02-11T23:31:16.772+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 69.607360840s delta_ub 1.980925322s) -- 0x3e02600 con 0x396b400 2026-02-11T23:31:16.772+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 69.607360840s delta_ub 1.980925322s) -- 0x3e02000 con 0x3752400 2026-02-11T23:31:16.772+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 69.607360840s delta_ub 4.649148464s) -- 0x3e02480 con 0x396b800 2026-02-11T23:31:16.772+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 69.607360840s delta_ub 4.649148464s) -- 0x3e03080 con 0x34c1000 2026-02-11T23:31:16.772+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 69.607360840s delta_ub 7.310500145s) -- 0x3da3800 con 0x3740000 2026-02-11T23:31:16.772+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 69.607360840s delta_ub 7.310500145s) -- 0x3da3080 con 0x3740400 2026-02-11T23:31:16.772+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 69.607360840s delta_ub 9.887018204s) -- 0x3da3500 con 0x3740800 2026-02-11T23:31:16.772+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e107 up_from 5 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 69.607360840s delta_ub 9.887018204s) -- 0x3da2780 con 0x3740c00 2026-02-11T23:31:16.772+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 23 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 67.626609802s delta_ub -1.980754614s) ==== 2033+0+0 (crc 0 0 0) 0x3e03680 con 0x3752400 2026-02-11T23:31:16.772+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:16.772+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:16.772+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:16.772+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:16.772+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980918169s,-1.980754614s]) 2026-02-11T23:31:16.772+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 23 ==== osd_ping(ping_reply e107 up_from 9 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 67.626632690s delta_ub -1.980729818s) ==== 2033+0+0 (crc 0 0 0) 0x3da2780 con 0x396b400 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.1 from_epoch 107 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:16.772+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980914831s,-1.980729818s]) 2026-02-11T23:31:16.772+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 22 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 64.958488464s delta_ub -4.648870945s) ==== 2033+0+0 (crc 0 0 0) 0x3e03680 con 0x34c1000 2026-02-11T23:31:16.772+0000 7fac71bfd640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:16.772+0000 7fac71bfd640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:16.772+0000 7fac71bfd640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:16.772+0000 7fac71bfd640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:16.772+0000 7fac71bfd640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649068356s,-4.648870945s]) 2026-02-11T23:31:16.772+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 20 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 62.297153473s delta_ub -7.310208321s) ==== 2033+0+0 (crc 0 0 0) 0x3da2780 con 0x3740000 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:16.772+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310433865s,-7.310208321s]) 2026-02-11T23:31:16.772+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 22 ==== osd_ping(ping_reply e107 up_from 13 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 64.958457947s delta_ub -4.648900032s) ==== 2033+0+0 (crc 0 0 0) 0x3da3500 con 0x396b800 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.2 from_epoch 107 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:16.772+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649144173s,-4.648900032s]) 2026-02-11T23:31:16.772+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 20 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 59.720626831s delta_ub -9.886732101s) ==== 2033+0+0 (crc 0 0 0) 0x3da2780 con 0x3740c00 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:16.772+0000 7fac703fa640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:16.772+0000 7fac703fa640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886981010s,-9.886732101s]) 2026-02-11T23:31:16.772+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 20 ==== osd_ping(ping_reply e107 up_from 17 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 62.297103882s delta_ub -7.310255527s) ==== 2033+0+0 (crc 0 0 0) 0x3da3500 con 0x3740400 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.3 from_epoch 107 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:16.772+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310515404s,-7.310255527s]) 2026-02-11T23:31:16.772+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 20 ==== osd_ping(ping_reply e107 up_from 21 ping_stamp 2026-02-11T23:31:16.772944+0000/69.607360840s send_stamp 59.720600128s delta_ub -9.886756897s) ==== 2033+0+0 (crc 0 0 0) 0x3da3500 con 0x3740800 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 get_con_osd_cluster to osd.4 from_epoch 107 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 get_nextmap_reserved map_reservations: {107=1} 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 release_map epoch: 107 2026-02-11T23:31:16.772+0000 7fac723fe640 10 osd.0 107 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 107 is not newer than session's projected_epoch of 107 2026-02-11T23:31:16.772+0000 7fac723fe640 20 osd.0 107 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887035370s,-9.886756897s]) 2026-02-11T23:31:16.881+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 55 ==== osd_map(108..108 src has 1..108) ==== 618+0+0 (crc 0 0 0) 0x46eafc0 con 0x396bc00 2026-02-11T23:31:16.881+0000 7fac626e8640 20 osd.0 107 OSD::ms_dispatch: osd_map(108..108 src has 1..108) 2026-02-11T23:31:16.881+0000 7fac626e8640 20 osd.0 107 _dispatch 0x46eafc0 osd_map(108..108 src has 1..108) 2026-02-11T23:31:16.881+0000 7fac626e8640 3 osd.0 107 handle_osd_map epochs [108,108], i have 107, src has [1,108] 2026-02-11T23:31:16.881+0000 7fac626e8640 10 osd.0 107 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:16.881+0000 7fac626e8640 10 osd.0 107 handle_osd_map got inc map for epoch 108 2026-02-11T23:31:16.881+0000 7fac626e8640 20 osd.0 107 got_full_map 108, nothing requested 2026-02-11T23:31:16.881+0000 7fac626e8640 20 osd.0 107 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:16.881+0000 7fac626e8640 20 osd.0 107 handle_osd_map pg_num_history pg_num_history(e108 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:31:16.881+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {108={}} 2026-02-11T23:31:16.881+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:16.881+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e108 maps [1~108] lci=[5,108] tlb=1) 2026-02-11T23:31:16.882+0000 7fac5e983640 10 osd.0 107 _committed_osd_maps 108..108 2026-02-11T23:31:16.882+0000 7fac5e983640 10 osd.0 107 advance to epoch 108 (<= last 108 <= newest_map 108) 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0 108 consume_map version 108 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0 107 await_reserved_maps epoch:108 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0 107 await_reserved_maps done 0 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0 108 consume_map 108 -- publish done 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x25d8f20 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 4.0 has no pg and waiting_for_split 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0 108 prune_pg_created 2026-02-11T23:31:16.882+0000 7fac5e983640 10 osd.0:0.consume_map 108 (was 107) 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:16.882+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 empty, pruning 2026-02-11T23:31:16.882+0000 7fac5e983640 10 osd.0:1.consume_map 108 (was 107) 2026-02-11T23:31:16.882+0000 7fac5e983640 10 osd.0:2.consume_map 108 (was 107) 2026-02-11T23:31:16.882+0000 7fac5e983640 10 osd.0:3.consume_map 108 (was 107) 2026-02-11T23:31:16.882+0000 7fac5e983640 10 osd.0:4.consume_map 108 (was 107) 2026-02-11T23:31:16.882+0000 7fac5e983640 10 osd.0:5.consume_map 108 (was 107) 2026-02-11T23:31:16.882+0000 7fac5e983640 10 osd.0:6.consume_map 108 (was 107) 2026-02-11T23:31:16.882+0000 7fac5e983640 10 osd.0:7.consume_map 108 (was 107) 2026-02-11T23:31:16.882+0000 7fac5e983640 10 osd.0 108 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:16.882+0000 7fac5e983640 7 osd.0 108 activate_map version 108 2026-02-11T23:31:16.882+0000 7fac55971640 20 osd.0 108 agent_entry empty queue 2026-02-11T23:31:16.883+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 29 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 108) e108/108) ==== 158+0+0 (unknown 0 0 0) 0x3f3f680 con 0x3752000 2026-02-11T23:31:16.883+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 28 ==== osd_map(108..108 src has 1..108) ==== 618+0+0 (unknown 0 0 0) 0x3941880 con 0x3752000 2026-02-11T23:31:16.883+0000 7fac606e4640 20 osd.0 108 OSD::ms_dispatch: osd_map(108..108 src has 1..108) 2026-02-11T23:31:16.883+0000 7fac606e4640 20 osd.0 108 _dispatch 0x3941880 osd_map(108..108 src has 1..108) 2026-02-11T23:31:16.883+0000 7fac703fa640 15 osd.0 108 enqueue_peering_evt 1.0 epoch_sent: 108 epoch_requested: 108 MQuery 1.0 from 1 query_epoch 108 query: query(info 0'0 epoch_sent 108) 2026-02-11T23:31:16.883+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 108 epoch_requested: 108 MQuery 1.0 from 1 query_epoch 108 query: query(info 0'0 epoch_sent 108)) class_id 2 prio 255 cost 10 e108) 2026-02-11T23:31:16.883+0000 7fac606e4640 3 osd.0 108 handle_osd_map epochs [108,108], i have 108, src has [1,108] 2026-02-11T23:31:16.883+0000 7fac606e4640 10 osd.0 108 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:16.883+0000 7fac606e4640 10 osd.0 108 no new maps here, dropping 2026-02-11T23:31:16.883+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:16.883+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 108 epoch_requested: 108 MQuery 1.0 from 1 query_epoch 108 query: query(info 0'0 epoch_sent 108)) class_id 2immediate prio 255 cost 10 e108) queued 2026-02-11T23:31:16.883+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:16.883+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 108 epoch_requested: 108 MQuery 1.0 from 1 query_epoch 108 query: query(info 0'0 epoch_sent 108)) class_id 2immediate prio 255 cost 10 e108) pg 0 2026-02-11T23:31:16.883+0000 7fac51168640 10 osd.0 108 handle_pg_query_nopg 1.0 2026-02-11T23:31:16.883+0000 7fac51168640 10 osd.0 108 pg 1.0 dne 2026-02-11T23:31:16.883+0000 7fac51168640 20 osd.0 108 get_con_osd_cluster to osd.1 from_epoch 108 2026-02-11T23:31:16.883+0000 7fac51168640 20 osd.0 108 get_nextmap_reserved map_reservations: {108=1} 2026-02-11T23:31:16.883+0000 7fac51168640 20 osd.0 108 release_map epoch: 108 2026-02-11T23:31:16.883+0000 7fac51168640 10 osd.0 108 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 107 -> 108 (shared) 2026-02-11T23:31:16.883+0000 7fac51168640 10 osd.0 108 send_incremental_map epoch range: (108, 108] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:31:16.883+0000 7fac51168640 10 osd.0 108 add_map_inc_bl 108 574 bytes 2026-02-11T23:31:16.883+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(108..108 src has 1..108) -- 0x38e2700 con 0x3752000 2026-02-11T23:31:16.883+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:108 sent:108 1.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e108/108) -- 0x486a300 con 0x3752000 2026-02-11T23:31:16.883+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:16.883+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:16.983+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 30 ==== osd_map(109..109 src has 1..109) ==== 651+0+0 (unknown 0 0 0) 0x38e2700 con 0x3752000 2026-02-11T23:31:16.983+0000 7fac606e4640 20 osd.0 108 OSD::ms_dispatch: osd_map(109..109 src has 1..109) 2026-02-11T23:31:16.983+0000 7fac606e4640 20 osd.0 108 _dispatch 0x38e2700 osd_map(109..109 src has 1..109) 2026-02-11T23:31:16.983+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 31 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 109) e109/109) ==== 158+0+0 (unknown 0 0 0) 0x3f3f680 con 0x3752000 2026-02-11T23:31:16.983+0000 7fac703fa640 15 osd.0 108 enqueue_peering_evt 1.0 epoch_sent: 109 epoch_requested: 109 MQuery 1.0 from 1 query_epoch 109 query: query(info 0'0 epoch_sent 109) 2026-02-11T23:31:16.983+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 MQuery 1.0 from 1 query_epoch 109 query: query(info 0'0 epoch_sent 109)) class_id 2 prio 255 cost 10 e109) 2026-02-11T23:31:16.983+0000 7fac606e4640 3 osd.0 108 handle_osd_map epochs [109,109], i have 108, src has [1,109] 2026-02-11T23:31:16.983+0000 7fac606e4640 10 osd.0 108 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:16.983+0000 7fac606e4640 10 osd.0 108 handle_osd_map got inc map for epoch 109 2026-02-11T23:31:16.983+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:16.983+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 MQuery 1.0 from 1 query_epoch 109 query: query(info 0'0 epoch_sent 109)) class_id 2immediate prio 255 cost 10 e109) queued 2026-02-11T23:31:16.983+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:16.983+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 MQuery 1.0 from 1 query_epoch 109 query: query(info 0'0 epoch_sent 109)) class_id 2immediate prio 255 cost 10 e109) pg 0 2026-02-11T23:31:16.983+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 map 109 > 108 2026-02-11T23:31:16.983+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 109, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 MQuery 1.0 from 1 query_epoch 109 query: query(info 0'0 epoch_sent 109)) class_id 2immediate prio 255 cost 10 e109) 2026-02-11T23:31:16.983+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:16.983+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:16.983+0000 7fac606e4640 20 osd.0 108 got_full_map 109, nothing requested 2026-02-11T23:31:16.983+0000 7fac606e4640 20 osd.0 108 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:16.983+0000 7fac606e4640 20 osd.0 108 handle_osd_map pg_num_history pg_num_history(e109 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:31:16.983+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {109={}} 2026-02-11T23:31:16.983+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:16.983+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e109 maps [1~109] lci=[5,109] tlb=1) 2026-02-11T23:31:16.984+0000 7fac5e983640 10 osd.0 108 _committed_osd_maps 109..109 2026-02-11T23:31:16.984+0000 7fac5e983640 10 osd.0 108 advance to epoch 109 (<= last 109 <= newest_map 109) 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0 109 consume_map version 109 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0 108 await_reserved_maps epoch:109 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0 108 await_reserved_maps done 0 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0 109 consume_map 109 -- publish done 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x25809a0 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 1.0 has no pg and waiting_for_split 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0 109 prune_pg_created 2026-02-11T23:31:16.984+0000 7fac5e983640 10 osd.0:0.consume_map 109 (was 108) 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 109 <= 109, requeueing 2026-02-11T23:31:16.984+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {109=} 2026-02-11T23:31:16.984+0000 7fac5e983640 10 osd.0:1.consume_map 109 (was 108) 2026-02-11T23:31:16.984+0000 7fac5e983640 10 osd.0:2.consume_map 109 (was 108) 2026-02-11T23:31:16.984+0000 7fac5e983640 10 osd.0:3.consume_map 109 (was 108) 2026-02-11T23:31:16.984+0000 7fac5e983640 10 osd.0:4.consume_map 109 (was 108) 2026-02-11T23:31:16.984+0000 7fac5e983640 10 osd.0:5.consume_map 109 (was 108) 2026-02-11T23:31:16.984+0000 7fac5e983640 10 osd.0:6.consume_map 109 (was 108) 2026-02-11T23:31:16.984+0000 7fac5e983640 10 osd.0:7.consume_map 109 (was 108) 2026-02-11T23:31:16.984+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:16.984+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 MQuery 1.0 from 1 query_epoch 109 query: query(info 0'0 epoch_sent 109)) class_id 2immediate prio 255 cost 10 e109) queued 2026-02-11T23:31:16.984+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:16.984+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 109 epoch_requested: 109 MQuery 1.0 from 1 query_epoch 109 query: query(info 0'0 epoch_sent 109)) class_id 2immediate prio 255 cost 10 e109) pg 0 2026-02-11T23:31:16.984+0000 7fac55170640 10 osd.0 109 handle_pg_query_nopg 1.0 2026-02-11T23:31:16.984+0000 7fac55170640 10 osd.0 109 pg 1.0 dne 2026-02-11T23:31:16.984+0000 7fac55170640 20 osd.0 109 get_con_osd_cluster to osd.1 from_epoch 109 2026-02-11T23:31:16.984+0000 7fac55170640 20 osd.0 109 get_nextmap_reserved map_reservations: {109=1} 2026-02-11T23:31:16.984+0000 7fac55170640 20 osd.0 109 release_map epoch: 109 2026-02-11T23:31:16.984+0000 7fac55170640 10 osd.0 109 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 108 -> 109 (shared) 2026-02-11T23:31:16.984+0000 7fac55170640 10 osd.0 109 send_incremental_map epoch range: (109, 109] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:31:16.984+0000 7fac55170640 10 osd.0 109 add_map_inc_bl 109 607 bytes 2026-02-11T23:31:16.984+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(109..109 src has 1..109) -- 0x46ea380 con 0x3752000 2026-02-11T23:31:16.984+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:109 sent:109 1.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e109/109) -- 0x3dc2700 con 0x3752000 2026-02-11T23:31:16.984+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:16.984+0000 7fac5e983640 10 osd.0 109 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:16.984+0000 7fac5e983640 7 osd.0 109 activate_map version 109 2026-02-11T23:31:16.984+0000 7fac55971640 20 osd.0 109 agent_entry empty queue 2026-02-11T23:31:17.083+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 32 ==== PGlog(1.0 log log((107'590,107'600], crt=107'600) pi ([103,108] all_participants={0, 1} intervals=[([103,107] acting={1})]) pg_lease(ru 83.937171936s ub 83.937171936s int 16.000000000s) e110/109) ==== 39009+0+0 (unknown 0 0 0) 0x3902800 con 0x3752000 2026-02-11T23:31:17.083+0000 7fac703fa640 15 osd.0 109 enqueue_peering_evt 1.0 epoch_sent: 110 epoch_requested: 109 MLogRec from 1 log log((107'590,107'600], crt=107'600) pi ([103,108] all_participants={0, 1} intervals=[([103,107] acting={1})]) pg_lease(ru 83.937171936s ub 83.937171936s int 16.000000000s) +create_info 2026-02-11T23:31:17.083+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 109 MLogRec from 1 log log((107'590,107'600], crt=107'600) pi ([103,108] all_participants={0, 1} intervals=[([103,107] acting={1})]) pg_lease(ru 83.937171936s ub 83.937171936s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e110) 2026-02-11T23:31:17.083+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.083+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 109 MLogRec from 1 log log((107'590,107'600], crt=107'600) pi ([103,108] all_participants={0, 1} intervals=[([103,107] acting={1})]) pg_lease(ru 83.937171936s ub 83.937171936s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e110) queued 2026-02-11T23:31:17.083+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.083+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 109 MLogRec from 1 log log((107'590,107'600], crt=107'600) pi ([103,108] all_participants={0, 1} intervals=[([103,107] acting={1})]) pg_lease(ru 83.937171936s ub 83.937171936s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e110) pg 0 2026-02-11T23:31:17.083+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 map 110 > 109 2026-02-11T23:31:17.083+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 110, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 109 MLogRec from 1 log log((107'590,107'600], crt=107'600) pi ([103,108] all_participants={0, 1} intervals=[([103,107] acting={1})]) pg_lease(ru 83.937171936s ub 83.937171936s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e110) 2026-02-11T23:31:17.083+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.083+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.091+0000 7fac68beb640 10 osd.0 109 tick 2026-02-11T23:31:17.091+0000 7fac68beb640 20 osd.0 109 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:17.092+0000 7fac683ea640 10 osd.0 109 tick_without_osd_lock 2026-02-11T23:31:17.092+0000 7fac683ea640 0 osd.0 109 set_statfs fake total 3686400 adjust available 3686400 2026-02-11T23:31:17.092+0000 7fac683ea640 20 osd.0 109 tick_without_osd_lock max_waiting_epoch 110, requesting new map 2026-02-11T23:31:17.092+0000 7fac683ea640 10 monclient: _renew_subs 2026-02-11T23:31:17.092+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:17.092+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=110}) -- 0x44a6340 con 0x396bc00 2026-02-11T23:31:17.092+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:17.092+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.085 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:17.092+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:17.092, recovery is active?:false restrictions:<....> 2026-02-11T23:31:17.092+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:17.092+0000 7fac683ea640 10 osd.0 109 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:17.092+0000 7fac683ea640 20 osd.0 109 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:17.092+0000 7fac683ea640 10 osd.0 109 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:17.092+0000 7fac683ea640 20 osd.0 109 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:17.092+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 56 ==== osd_map(110..110 src has 1..110) ==== 296+0+0 (crc 0 0 0) 0x46eac40 con 0x396bc00 2026-02-11T23:31:17.092+0000 7fac626e8640 20 osd.0 109 OSD::ms_dispatch: osd_map(110..110 src has 1..110) 2026-02-11T23:31:17.092+0000 7fac626e8640 20 osd.0 109 _dispatch 0x46eac40 osd_map(110..110 src has 1..110) 2026-02-11T23:31:17.092+0000 7fac626e8640 3 osd.0 109 handle_osd_map epochs [110,110], i have 109, src has [1,110] 2026-02-11T23:31:17.092+0000 7fac626e8640 10 osd.0 109 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.092+0000 7fac626e8640 10 osd.0 109 handle_osd_map got inc map for epoch 110 2026-02-11T23:31:17.092+0000 7fac626e8640 20 osd.0 109 got_full_map 110, nothing requested 2026-02-11T23:31:17.092+0000 7fac626e8640 20 osd.0 109 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:17.092+0000 7fac626e8640 20 osd.0 109 handle_osd_map pg_num_history pg_num_history(e110 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:31:17.092+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {110={}} 2026-02-11T23:31:17.092+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:17.092+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e110 maps [1~110] lci=[5,110] tlb=1) 2026-02-11T23:31:17.093+0000 7fac5e983640 10 osd.0 109 _committed_osd_maps 110..110 2026-02-11T23:31:17.093+0000 7fac5e983640 10 osd.0 109 advance to epoch 110 (<= last 110 <= newest_map 110) 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0 110 consume_map version 110 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0 109 await_reserved_maps epoch:110 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0 109 await_reserved_maps done 0 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0 110 consume_map 110 -- publish done 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 1 slots 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x25809a0 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 1.0 has no pg and waiting_for_split 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0 110 prune_pg_created 2026-02-11T23:31:17.093+0000 7fac5e983640 10 osd.0:0.consume_map 110 (was 109) 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 110 <= 110, requeueing 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {110=} 2026-02-11T23:31:17.093+0000 7fac5e983640 10 osd.0:1.consume_map 110 (was 109) 2026-02-11T23:31:17.093+0000 7fac5e983640 10 osd.0:2.consume_map 110 (was 109) 2026-02-11T23:31:17.093+0000 7fac5e983640 10 osd.0:3.consume_map 110 (was 109) 2026-02-11T23:31:17.093+0000 7fac5e983640 10 osd.0:4.consume_map 110 (was 109) 2026-02-11T23:31:17.093+0000 7fac5e983640 10 osd.0:5.consume_map 110 (was 109) 2026-02-11T23:31:17.093+0000 7fac5e983640 10 osd.0:6.consume_map 110 (was 109) 2026-02-11T23:31:17.093+0000 7fac5e983640 10 osd.0:7.consume_map 110 (was 109) 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 109 MLogRec from 1 log log((107'590,107'600], crt=107'600) pi ([103,108] all_participants={0, 1} intervals=[([103,107] acting={1})]) pg_lease(ru 83.937171936s ub 83.937171936s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e110) queued 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 109 MLogRec from 1 log log((107'590,107'600], crt=107'600) pi ([103,108] all_participants={0, 1} intervals=[([103,107] acting={1})]) pg_lease(ru 83.937171936s ub 83.937171936s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e110) pg 0 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 no pg, should create on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 109 MLogRec from 1 log log((107'590,107'600], crt=107'600) pi ([103,108] all_participants={0, 1} intervals=[([103,107] acting={1})]) pg_lease(ru 83.937171936s ub 83.937171936s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e110) 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 110 _make_pg 1.0 2026-02-11T23:31:17.093+0000 7fac51168640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:31:17.093+0000 7fac51168640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:31:17.093+0000 7fac51168640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {"0000000000000001."}] update_bits prefix updated 2026-02-11T23:31:17.093+0000 7fac51168640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:1, shard:-1, shard_prefix: , prefixes: {"0000000000000001."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:31:17.093+0000 7fac51168640 5 osd.0 pg_epoch: 109 pg[1.0(unlocked)] enter Initial 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 pg_epoch: 109 pg[1.0(unlocked)] enter NotTrimming 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role -1 up [1,0] acting [1] history ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109 past_intervals ([103,108] all_participants={0, 1} intervals=[([103,107] acting={1})]) 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [1]+[1,0] 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 69.928092957s + 0.000000000s) 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:31:17.093+0000 7fac51168640 15 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:17.093+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:17.093+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:17.093+0000 7fac51168640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:31:17.093+0000 7fac51168640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:31:17.093+0000 7fac51168640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:31:17.093+0000 7fac51168640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:31:17.093+0000 7fac51168640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:31:17.093+0000 7fac51168640 10 end of _write_log_and_missing 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:31:17.093+0000 7fac51168640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] exit Initial 0.000075 0 0.000000 2026-02-11T23:31:17.093+0000 7fac51168640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=0 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 109 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 109..109 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.093+0000 7fac51168640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] exit Reset 0.000017 1 0.000016 2026-02-11T23:31:17.093+0000 7fac51168640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:31:17.093+0000 7fac51168640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:31:17.093+0000 7fac51168640 1 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:31:17.093+0000 7fac51168640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:31:17.093+0000 7fac51168640 5 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 109 while current is 109 2026-02-11T23:31:17.093+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 110 handle_pg_create_info new pg pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0:0._attach_pg 1.0 0x4a2a000 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 110 identify_splits_and_merges 1.0 e109 to e110 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 110 identify_splits_and_merges 1.0 e109 to e110 pg_nums {24=1} 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0:0._prime_splits 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] handle_advance_map: 110 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 109 -> 110 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 109 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1] -- 1/1 2026-02-11T23:31:17.093+0000 7fac5e983640 10 osd.0 110 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:31:17.093+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.093+0000 7fac5e983640 15 osd.0 110 enqueue_peering_evt 1.0 epoch_sent: 110 epoch_requested: 110 NullEvt 2026-02-11T23:31:17.093+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 NullEvt) class_id 2 prio 255 cost 10 e110) 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 110 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.093+0000 7fac5e983640 7 osd.0 110 activate_map version 110 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 110..110 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 109 while current is 110 2026-02-11T23:31:17.093+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.093+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 110 epoch_requested: 109 MLogRec from 1 log log((107'590,107'600], crt=107'600) pi ([103,108] all_participants={0, 1} intervals=[([103,107] acting={1})]) pg_lease(ru 83.937171936s ub 83.937171936s int 16.000000000s) +create_info 2026-02-11T23:31:17.093+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 NullEvt) class_id 2immediate prio 255 cost 10 e110) queued 2026-02-11T23:31:17.093+0000 7fac55971640 20 osd.0 110 agent_entry empty queue 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( empty local-lis/les=0/0 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=0'0 unknown mbc={}] state: got info+log from osd.1 1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) log((107'590,107'600], crt=107'600) 2026-02-11T23:31:17.093+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] PeeringState::proc_lease no-op, !nonprimary 2026-02-11T23:31:17.093+0000 7fac51168640 5 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] exit Started/Stray 0.000251 3 0.000015 2026-02-11T23:31:17.093+0000 7fac51168640 5 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:31:17.093+0000 7fac51168640 5 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 107'600 -> 107'600 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:31:17.093+0000 7fac51168640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:31:17.093+0000 7fac51168640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:31:17.093+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:31:17.093+0000 7fac51168640 6 write_log_and_missing with: dirty_to: 4294967295'18446744073709551615, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:31:17.093+0000 7fac51168640 10 _write_log_and_missing clearing up to 4294967295'18446744073709551615 dirty_to_dups=4294967295'18446744073709551615 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:31:17.093+0000 7fac51168640 10 _write_log_and_missing remove dups min=dup_0000000000.00000000000000000000 to dirty_to_dup=dup_4294967295.18446744073709551615 2026-02-11T23:31:17.093+0000 7fac51168640 10 _write_log_and_missing going to encode log.dups.size()=590 2026-02-11T23:31:17.094+0000 7fac51168640 10 _write_log_and_missing 1st round encoded log.dups.size()=590 2026-02-11T23:31:17.094+0000 7fac51168640 10 _write_log_and_missing 2st round encoded log.dups.size()=590 2026-02-11T23:31:17.094+0000 7fac51168640 10 end of _write_log_and_missing 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 NullEvt) class_id 2immediate prio 255 cost 10 e110) pg 0x4a2a000 2026-02-11T23:31:17.095+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] do_peering_event: epoch_sent: 110 epoch_requested: 110 NullEvt 2026-02-11T23:31:17.095+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.095+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.095+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.095+0000 7fac55170640 15 osd.0 110 enqueue_peering_evt 1.0 epoch_sent: 110 epoch_requested: 110 ActivateCommitted from 110 processed at 110 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 ActivateCommitted from 110 processed at 110) class_id 2 prio 255 cost 10 e110) 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 ActivateCommitted from 110 processed at 110) class_id 2immediate prio 255 cost 10 e110) queued 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 ActivateCommitted from 110 processed at 110) class_id 2immediate prio 255 cost 10 e110) pg 0x4a2a000 2026-02-11T23:31:17.095+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.095+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] do_peering_event: epoch_sent: 110 epoch_requested: 110 ActivateCommitted from 110 processed at 110 2026-02-11T23:31:17.095+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 crt=107'600 lcod 0'0 unknown mbc={}] state: react 110 telling primary 2026-02-11T23:31:17.095+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.095+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 109 -> 110 (shared) 2026-02-11T23:31:17.095+0000 7fac55170640 10 osd.0 110 send_incremental_map epoch range: (110, 110] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:31:17.095+0000 7fac55170640 10 osd.0 110 add_map_inc_bl 110 252 bytes 2026-02-11T23:31:17.095+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(110..110 src has 1..110) -- 0x3940700 con 0x3752000 2026-02-11T23:31:17.095+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_info2(1.0 1.0 1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) e110/110) -- 0x3dc2e00 con 0x3752000 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.095+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 33 ==== pg_info2(1.0 1.0 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) e110/110) ==== 1111+0+0 (unknown 0 0 0) 0x3dc2e00 con 0x3752000 2026-02-11T23:31:17.095+0000 7fac703fa640 15 osd.0 110 enqueue_peering_evt 1.0 epoch_sent: 110 epoch_requested: 110 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) pg_lease(ru 83.949714661s ub 83.949714661s int 16.000000000s) 2026-02-11T23:31:17.095+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) pg_lease(ru 83.949714661s ub 83.949714661s int 16.000000000s)) class_id 2 prio 255 cost 10 e110) 2026-02-11T23:31:17.095+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.095+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) pg_lease(ru 83.949714661s ub 83.949714661s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e110) queued 2026-02-11T23:31:17.095+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.095+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) pg_lease(ru 83.949714661s ub 83.949714661s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e110) pg 0x4a2a000 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.095+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=103/103 les/c/f=105/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 110 epoch_requested: 110 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) pg_lease(ru 83.949714661s ub 83.949714661s int 16.000000000s) 2026-02-11T23:31:17.095+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] PeeringState::update_history advanced history from ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109 2026-02-11T23:31:17.095+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.095+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.095+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.095+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.095+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 34 ==== MBackfillReserve(1.0 REQUEST prio: 151 e110/110) ==== 46+0+0 (unknown 0 0 0) 0x3da2780 con 0x3752000 2026-02-11T23:31:17.095+0000 7fac703fa640 15 osd.0 110 enqueue_peering_evt 1.0 epoch_sent: 110 epoch_requested: 110 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0 2026-02-11T23:31:17.095+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2 prio 255 cost 10 e110) 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2immediate prio 255 cost 10 e110) queued 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2immediate prio 255 cost 10 e110) pg 0x4a2a000 2026-02-11T23:31:17.096+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.096+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 110 epoch_requested: 110 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0 2026-02-11T23:31:17.096+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] try_reserve_recovery_space primary_bytes 2400KiB local 0KiB pending_adjustments 2400KiB 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] _tentative_full type backfillfull adjust_used 2400KiB 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 110 compute_adjusted_ratio Before kb_used() 0 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 110 compute_adjusted_ratio After kb_used() 2400 2026-02-11T23:31:17.096+0000 7fac55170640 5 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.002449 4 0.000223 2026-02-11T23:31:17.096+0000 7fac55170640 5 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] enter Started/ReplicaActive/RepWaitBackfillReserved 2026-02-11T23:31:17.096+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.096+0000 7fac58176640 15 osd.0 110 enqueue_peering_evt 1.0 epoch_sent: 110 epoch_requested: 110 RemoteBackfillReserved 2026-02-11T23:31:17.096+0000 7fac58176640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 RemoteBackfillReserved) class_id 2 prio 255 cost 10 e110) 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 RemoteBackfillReserved) class_id 2immediate prio 255 cost 10 e110) queued 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 110 epoch_requested: 110 RemoteBackfillReserved) class_id 2immediate prio 255 cost 10 e110) pg 0x4a2a000 2026-02-11T23:31:17.096+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 110 epoch_requested: 110 RemoteBackfillReserved 2026-02-11T23:31:17.096+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.096+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MBackfillReserve(1.0 GRANT e110/110) -- 0x3e03200 con 0x3752000 2026-02-11T23:31:17.096+0000 7fac55170640 5 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] exit Started/ReplicaActive/RepWaitBackfillReserved 0.000025 1 0.000022 2026-02-11T23:31:17.096+0000 7fac55170640 5 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] enter Started/ReplicaActive/RepRecovering 2026-02-11T23:31:17.096+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.096+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.098+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 36 ==== pg_scan(get_digest 1.0 MIN-MIN e 110/109) ==== 119+0+0 (unknown 0 0 0) 0x38eaa00 con 0x3752000 2026-02-11T23:31:17.098+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_scan(get_digest 1.0 MIN-MIN e 110/109) v2 prio 15 type 94 cost 0 latency 0.000008 epoch 110 2026-02-11T23:31:17.098+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 35 ==== osd_map(110..110 src has 1..110) ==== 296+0+0 (unknown 0 0 0) 0x3940700 con 0x3752000 2026-02-11T23:31:17.098+0000 7fac606e4640 20 osd.0 110 OSD::ms_dispatch: osd_map(110..110 src has 1..110) 2026-02-11T23:31:17.098+0000 7fac606e4640 20 osd.0 110 _dispatch 0x3940700 osd_map(110..110 src has 1..110) 2026-02-11T23:31:17.098+0000 7fac606e4640 3 osd.0 110 handle_osd_map epochs [110,110], i have 110, src has [1,110] 2026-02-11T23:31:17.098+0000 7fac606e4640 10 osd.0 110 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.098+0000 7fac606e4640 10 osd.0 110 no new maps here, dropping 2026-02-11T23:31:17.098+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_scan(get_digest 1.0 MIN-MIN e 110/109) v2) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.098+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.098+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_scan(get_digest 1.0 MIN-MIN e 110/109) v2) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.098+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.098+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_scan(get_digest 1.0 MIN-MIN e 110/109) v2) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.098+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.098+0000 7fac51168640 10 osd.0 110 dequeue_op pg_scan(get_digest 1.0 MIN-MIN e 110/109) v2 prio 15 cost 0 latency 0.000032 pg_scan(get_digest 1.0 MIN-MIN e 110/109) v2 pg pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] 2026-02-11T23:31:17.098+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.098+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] _handle_message: pg_scan(get_digest 1.0 MIN-MIN e 110/109) v2 2026-02-11T23:31:17.098+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] do_scan pg_scan(get_digest 1.0 MIN-MIN e 110/109) 2026-02-11T23:31:17.098+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] scan_range_replica from MIN 2026-02-11T23:31:17.098+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] got 0 items, next MAX 2026-02-11T23:31:17.098+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] [] 2026-02-11T23:31:17.098+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_scan(digest 1.0 MIN-MAX e 110/109) -- 0x38eac80 con 0x3752000 2026-02-11T23:31:17.098+0000 7fac51168640 10 osd.0 110 dequeue_op pg_scan(get_digest 1.0 MIN-MIN e 110/109) v2 finish latency 0.000075 2026-02-11T23:31:17.098+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.099+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 37 ==== MOSDPGPush(1.0 110/109 [PushOp(1:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3752000 2026-02-11T23:31:17.099+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 110 2026-02-11T23:31:17.099+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.099+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.099+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 110/109 [PushOp(1:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] 2026-02-11T23:31:17.099+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.099+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.099+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] handle_push ObjectRecoveryInfo(1:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.099+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] submit_push_data 1:006bea6a:::obj172:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] submit_push_data recovering object 1:006bea6a:::obj172:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.099+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] on_local_recover: 1:006bea6a:::obj172:head 2026-02-11T23:31:17.099+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] PeeringState::recover_got got missing 1:006bea6a:::obj172:head v 107'172 2026-02-11T23:31:17.099+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.099+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.099+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.099+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000086 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.099+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.099+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.099+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x3e03c80 con 0x3752000 2026-02-11T23:31:17.099+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:006bea6a:::obj172:head)]) -- 0x3e03200 con 0x3752000 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.099+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 38 ==== MOSDPGPush(1.0 110/109 [PushOp(1:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x389aea0 con 0x3752000 2026-02-11T23:31:17.099+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.099+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.099+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.099+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 39 ==== pg_backfill(progress 1.0 e 110/110 lb 1:006bea6a:::obj172:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3752000 2026-02-11T23:31:17.099+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:006bea6a:::obj172:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.099+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.099+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:006bea6a:::obj172:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.099+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.099+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.099+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 110/109 [PushOp(1:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.099+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.099+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:006bea6a:::obj172:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.099+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.099+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.099+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:00833431:::obj76:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.099+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:00833431:::obj76:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.099+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:00833431:::obj76:head 2026-02-11T23:31:17.099+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:00833431:::obj76:head v 107'76 2026-02-11T23:31:17.099+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.099+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.100+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:17.100+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:006bea6a:::obj172:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:006bea6a:::obj172:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 1.0 e 110/110 lb 1:006bea6a:::obj172:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:006bea6a:::obj172:head) v3 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=109/110 n=0 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:006bea6a:::obj172:head) 2026-02-11T23:31:17.100+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:006bea6a:::obj172:head) v3 finish latency 0.000014 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.100+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.100+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.100+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x3e03e00 con 0x3752000 2026-02-11T23:31:17.100+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:00833431:::obj76:head)]) -- 0x4af2000 con 0x3752000 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.100+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 40 ==== MOSDPGPush(1.0 110/109 [PushOp(1:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x449da00 con 0x3752000 2026-02-11T23:31:17.100+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.100+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.100+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.100+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 41 ==== pg_backfill(progress 1.0 e 110/110 lb 1:00833431:::obj76:head) ==== 1061+0+0 (unknown 0 0 0) 0x48bb080 con 0x3752000 2026-02-11T23:31:17.100+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.100+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:00833431:::obj76:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.100+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:00833431:::obj76:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.100+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.100+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.100+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 110/109 [PushOp(1:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.100+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:00833431:::obj76:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.100+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.100+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.100+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:00b31b6c:::obj19:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.100+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:00b31b6c:::obj19:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.100+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:00b31b6c:::obj19:head 2026-02-11T23:31:17.100+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:00b31b6c:::obj19:head v 107'19 2026-02-11T23:31:17.100+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.100+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.100+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.100+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:00833431:::obj76:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:00833431:::obj76:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 110/110 lb 1:00833431:::obj76:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:00833431:::obj76:head) v3 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:006bea6a:::obj172:head local-lis/les=109/110 n=1 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:00833431:::obj76:head) 2026-02-11T23:31:17.100+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:00833431:::obj76:head) v3 finish latency 0.000013 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.100+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.100+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.100+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2480 con 0x3752000 2026-02-11T23:31:17.100+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:00b31b6c:::obj19:head)]) -- 0x4af2180 con 0x3752000 2026-02-11T23:31:17.100+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.101+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 42 ==== MOSDPGPush(1.0 110/109 [PushOp(1:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3e56340 con 0x3752000 2026-02-11T23:31:17.101+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.101+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.101+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.101+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 43 ==== pg_backfill(progress 1.0 e 110/110 lb 1:00b31b6c:::obj19:head) ==== 1061+0+0 (unknown 0 0 0) 0x40ecc00 con 0x3752000 2026-02-11T23:31:17.101+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:00b31b6c:::obj19:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.101+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.101+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:00b31b6c:::obj19:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.101+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.101+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.101+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.101+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:00b31b6c:::obj19:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.101+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 110/109 [PushOp(1:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.101+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.101+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.101+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.101+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:010e1e72:::obj351:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.101+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:010e1e72:::obj351:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.101+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:010e1e72:::obj351:head 2026-02-11T23:31:17.101+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:010e1e72:::obj351:head v 107'351 2026-02-11T23:31:17.101+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.101+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.101+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.101+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.101+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.101+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:00b31b6c:::obj19:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.101+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:00b31b6c:::obj19:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 110/110 lb 1:00b31b6c:::obj19:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.101+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.101+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:00b31b6c:::obj19:head) v3 2026-02-11T23:31:17.101+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00833431:::obj76:head local-lis/les=109/110 n=2 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:00b31b6c:::obj19:head) 2026-02-11T23:31:17.101+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.101+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:00b31b6c:::obj19:head) v3 finish latency 0.000011 2026-02-11T23:31:17.101+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.101+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.101+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.101+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.101+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.101+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.101+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.101+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2600 con 0x3752000 2026-02-11T23:31:17.101+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:010e1e72:::obj351:head)]) -- 0x4af2300 con 0x3752000 2026-02-11T23:31:17.101+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.102+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 44 ==== MOSDPGPush(1.0 110/109 [PushOp(1:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44a64e0 con 0x3752000 2026-02-11T23:31:17.102+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 110 2026-02-11T23:31:17.102+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.102+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.102+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 45 ==== pg_backfill(progress 1.0 e 110/110 lb 1:010e1e72:::obj351:head) ==== 1062+0+0 (unknown 0 0 0) 0x40ed180 con 0x3752000 2026-02-11T23:31:17.102+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.102+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:010e1e72:::obj351:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.102+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:010e1e72:::obj351:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.102+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.102+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 110/109 [PushOp(1:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:010e1e72:::obj351:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:015e5cd2:::obj354:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.102+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:015e5cd2:::obj354:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:015e5cd2:::obj354:head 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:015e5cd2:::obj354:head v 107'354 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.102+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.102+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:010e1e72:::obj351:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.102+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:010e1e72:::obj351:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 110/110 lb 1:010e1e72:::obj351:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.102+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.102+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:010e1e72:::obj351:head) v3 2026-02-11T23:31:17.102+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:00b31b6c:::obj19:head local-lis/les=109/110 n=3 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:010e1e72:::obj351:head) 2026-02-11T23:31:17.102+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.102+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:010e1e72:::obj351:head) v3 finish latency 0.000011 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.102+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.102+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.102+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2780 con 0x3752000 2026-02-11T23:31:17.102+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:015e5cd2:::obj354:head)]) -- 0x4af2c00 con 0x3752000 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.102+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 46 ==== MOSDPGPush(1.0 110/109 [PushOp(1:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3e56b60 con 0x3752000 2026-02-11T23:31:17.102+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.102+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.102+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.102+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 47 ==== pg_backfill(progress 1.0 e 110/110 lb 1:015e5cd2:::obj354:head) ==== 1062+0+0 (unknown 0 0 0) 0x40ea580 con 0x3752000 2026-02-11T23:31:17.102+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.102+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:015e5cd2:::obj354:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.102+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:015e5cd2:::obj354:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.102+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.102+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.102+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:015e5cd2:::obj354:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.102+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:022ea2cc:::obj392:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.103+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:022ea2cc:::obj392:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:022ea2cc:::obj392:head 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:022ea2cc:::obj392:head v 107'392 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.103+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.103+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:015e5cd2:::obj354:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:015e5cd2:::obj354:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 110/110 lb 1:015e5cd2:::obj354:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:015e5cd2:::obj354:head) v3 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:010e1e72:::obj351:head local-lis/les=109/110 n=4 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:015e5cd2:::obj354:head) 2026-02-11T23:31:17.103+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:015e5cd2:::obj354:head) v3 finish latency 0.000010 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.103+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.103+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2900 con 0x3752000 2026-02-11T23:31:17.103+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:022ea2cc:::obj392:head)]) -- 0x4af2f00 con 0x3752000 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.103+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 48 ==== MOSDPGPush(1.0 110/109 [PushOp(1:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3e57040 con 0x3752000 2026-02-11T23:31:17.103+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.103+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.103+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.103+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 49 ==== pg_backfill(progress 1.0 e 110/110 lb 1:022ea2cc:::obj392:head) ==== 1062+0+0 (unknown 0 0 0) 0x40ed700 con 0x3752000 2026-02-11T23:31:17.103+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.103+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:022ea2cc:::obj392:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.103+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.103+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:022ea2cc:::obj392:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.103+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:022ea2cc:::obj392:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:02455f6c:::obj462:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.103+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:02455f6c:::obj462:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:02455f6c:::obj462:head 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:02455f6c:::obj462:head v 107'462 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.103+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.103+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.103+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:022ea2cc:::obj392:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:022ea2cc:::obj392:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 110/110 lb 1:022ea2cc:::obj392:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:022ea2cc:::obj392:head) v3 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:015e5cd2:::obj354:head local-lis/les=109/110 n=5 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:022ea2cc:::obj392:head) 2026-02-11T23:31:17.103+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:022ea2cc:::obj392:head) v3 finish latency 0.000010 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.103+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.103+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.103+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af3500 con 0x3752000 2026-02-11T23:31:17.103+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:02455f6c:::obj462:head)]) -- 0x4af3200 con 0x3752000 2026-02-11T23:31:17.103+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.104+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 50 ==== MOSDPGPush(1.0 110/109 [PushOp(1:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3e57520 con 0x3752000 2026-02-11T23:31:17.104+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.104+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.104+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.104+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 51 ==== pg_backfill(progress 1.0 e 110/110 lb 1:02455f6c:::obj462:head) ==== 1062+0+0 (unknown 0 0 0) 0x40eb080 con 0x3752000 2026-02-11T23:31:17.104+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:02455f6c:::obj462:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.104+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.104+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:02455f6c:::obj462:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.104+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.104+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.104+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.104+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:02455f6c:::obj462:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.104+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.104+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.104+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.104+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.104+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:024707f1:::obj490:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.104+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:024707f1:::obj490:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.104+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:024707f1:::obj490:head 2026-02-11T23:31:17.104+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:024707f1:::obj490:head v 107'490 2026-02-11T23:31:17.104+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.104+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.104+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.104+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.104+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.104+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:02455f6c:::obj462:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.104+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:02455f6c:::obj462:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 110/110 lb 1:02455f6c:::obj462:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.104+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.104+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:02455f6c:::obj462:head) v3 2026-02-11T23:31:17.104+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:022ea2cc:::obj392:head local-lis/les=109/110 n=6 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:02455f6c:::obj462:head) 2026-02-11T23:31:17.104+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.104+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:02455f6c:::obj462:head) v3 finish latency 0.000011 2026-02-11T23:31:17.104+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.104+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.104+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.104+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.104+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.104+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.104+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.104+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af3680 con 0x3752000 2026-02-11T23:31:17.104+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:024707f1:::obj490:head)]) -- 0x4af3c80 con 0x3752000 2026-02-11T23:31:17.104+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.105+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 52 ==== MOSDPGPush(1.0 110/109 [PushOp(1:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3e57a00 con 0x3752000 2026-02-11T23:31:17.105+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.105+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.105+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 53 ==== pg_backfill(progress 1.0 e 110/110 lb 1:024707f1:::obj490:head) ==== 1062+0+0 (unknown 0 0 0) 0x46e4100 con 0x3752000 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.105+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:024707f1:::obj490:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.105+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:024707f1:::obj490:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:024707f1:::obj490:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:02817247:::obj101:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:02817247:::obj101:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:02817247:::obj101:head 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:02817247:::obj101:head v 107'101 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.105+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:024707f1:::obj490:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:024707f1:::obj490:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 110/110 lb 1:024707f1:::obj490:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:024707f1:::obj490:head) v3 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02455f6c:::obj462:head local-lis/les=109/110 n=7 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:024707f1:::obj490:head) 2026-02-11T23:31:17.105+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:024707f1:::obj490:head) v3 finish latency 0.000010 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.105+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af3800 con 0x3752000 2026-02-11T23:31:17.105+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:02817247:::obj101:head)]) -- 0x4cb4000 con 0x3752000 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.105+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 54 ==== MOSDPGPush(1.0 110/109 [PushOp(1:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4cb6000 con 0x3752000 2026-02-11T23:31:17.105+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.105+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.105+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 55 ==== pg_backfill(progress 1.0 e 110/110 lb 1:02817247:::obj101:head) ==== 1062+0+0 (unknown 0 0 0) 0x46e2580 con 0x3752000 2026-02-11T23:31:17.105+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:02817247:::obj101:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.105+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:02817247:::obj101:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:02817247:::obj101:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:03553543:::obj227:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:03553543:::obj227:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:03553543:::obj227:head 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:03553543:::obj227:head v 107'227 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.105+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.105+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:02817247:::obj101:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:02817247:::obj101:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 110/110 lb 1:02817247:::obj101:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:02817247:::obj101:head) v3 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:024707f1:::obj490:head local-lis/les=109/110 n=8 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:02817247:::obj101:head) 2026-02-11T23:31:17.105+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:02817247:::obj101:head) v3 finish latency 0.000010 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.105+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.105+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.105+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af3980 con 0x3752000 2026-02-11T23:31:17.105+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:03553543:::obj227:head)]) -- 0x4cb4300 con 0x3752000 2026-02-11T23:31:17.105+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.106+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 56 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4cb64e0 con 0x3752000 2026-02-11T23:31:17.106+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.106+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.106+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.106+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 57 ==== pg_backfill(progress 1.0 e 110/110 lb 1:03553543:::obj227:head) ==== 1062+0+0 (unknown 0 0 0) 0x46e4680 con 0x3752000 2026-02-11T23:31:17.106+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.106+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:03553543:::obj227:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.106+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:03553543:::obj227:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.106+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.106+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.106+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.106+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.106+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.106+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:03553543:::obj227:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.106+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.106+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.106+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0366f813:::obj557:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.106+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0366f813:::obj557:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.106+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0366f813:::obj557:head 2026-02-11T23:31:17.106+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0366f813:::obj557:head v 107'557 2026-02-11T23:31:17.106+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.106+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.106+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.106+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.106+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.106+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:03553543:::obj227:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.106+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:03553543:::obj227:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 110/110 lb 1:03553543:::obj227:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.106+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.106+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:03553543:::obj227:head) v3 2026-02-11T23:31:17.106+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:02817247:::obj101:head local-lis/les=109/110 n=9 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:03553543:::obj227:head) 2026-02-11T23:31:17.106+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.106+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:03553543:::obj227:head) v3 finish latency 0.000010 2026-02-11T23:31:17.106+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.106+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.106+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.106+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.106+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.106+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.106+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.106+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af3b00 con 0x3752000 2026-02-11T23:31:17.106+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0366f813:::obj557:head)]) -- 0x4cb4600 con 0x3752000 2026-02-11T23:31:17.106+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.107+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 58 ==== MOSDPGPush(1.0 110/109 [PushOp(1:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4cb69c0 con 0x3752000 2026-02-11T23:31:17.107+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.107+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.107+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.107+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 59 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0366f813:::obj557:head) ==== 1062+0+0 (unknown 0 0 0) 0x46e5180 con 0x3752000 2026-02-11T23:31:17.107+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0366f813:::obj557:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.107+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.107+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0366f813:::obj557:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.107+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.107+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.107+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.107+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0366f813:::obj557:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.107+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.107+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.107+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.107+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.107+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:03b9100b:::obj283:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.107+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:03b9100b:::obj283:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.107+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:03b9100b:::obj283:head 2026-02-11T23:31:17.107+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:03b9100b:::obj283:head v 107'283 2026-02-11T23:31:17.107+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.107+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.107+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.107+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.107+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.107+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0366f813:::obj557:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.107+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0366f813:::obj557:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 110/110 lb 1:0366f813:::obj557:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.107+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.107+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0366f813:::obj557:head) v3 2026-02-11T23:31:17.107+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03553543:::obj227:head local-lis/les=109/110 n=10 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0366f813:::obj557:head) 2026-02-11T23:31:17.107+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.107+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0366f813:::obj557:head) v3 finish latency 0.000010 2026-02-11T23:31:17.107+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.107+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.107+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.107+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.107+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.107+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.107+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.107+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2000 con 0x3752000 2026-02-11T23:31:17.107+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:03b9100b:::obj283:head)]) -- 0x3e02c00 con 0x3752000 2026-02-11T23:31:17.107+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.107+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 60 ==== MOSDPGPush(1.0 110/109 [PushOp(1:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4cb6ea0 con 0x3752000 2026-02-11T23:31:17.108+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 110 2026-02-11T23:31:17.108+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.108+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 61 ==== pg_backfill(progress 1.0 e 110/110 lb 1:03b9100b:::obj283:head) ==== 1062+0+0 (unknown 0 0 0) 0x46e3b80 con 0x3752000 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.108+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:03b9100b:::obj283:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.108+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:03b9100b:::obj283:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:03b9100b:::obj283:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:040c73f9:::obj237:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:040c73f9:::obj237:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:040c73f9:::obj237:head 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:040c73f9:::obj237:head v 107'237 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.108+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:03b9100b:::obj283:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:03b9100b:::obj283:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:03b9100b:::obj283:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:03b9100b:::obj283:head) v3 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0366f813:::obj557:head local-lis/les=109/110 n=11 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:03b9100b:::obj283:head) 2026-02-11T23:31:17.108+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:03b9100b:::obj283:head) v3 finish latency 0.000010 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.108+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2480 con 0x3752000 2026-02-11T23:31:17.108+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:040c73f9:::obj237:head)]) -- 0x3e02600 con 0x3752000 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.108+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 62 ==== MOSDPGPush(1.0 110/109 [PushOp(1:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4cb7380 con 0x3752000 2026-02-11T23:31:17.108+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 110 2026-02-11T23:31:17.108+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.108+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 63 ==== pg_backfill(progress 1.0 e 110/110 lb 1:040c73f9:::obj237:head) ==== 1062+0+0 (unknown 0 0 0) 0x46e2000 con 0x3752000 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.108+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:040c73f9:::obj237:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.108+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:040c73f9:::obj237:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:040c73f9:::obj237:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:04510f17:::obj73:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:04510f17:::obj73:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:04510f17:::obj73:head 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:04510f17:::obj73:head v 107'73 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.108+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.108+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:040c73f9:::obj237:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:040c73f9:::obj237:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 110/110 lb 1:040c73f9:::obj237:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:040c73f9:::obj237:head) v3 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:03b9100b:::obj283:head local-lis/les=109/110 n=12 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:040c73f9:::obj237:head) 2026-02-11T23:31:17.108+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:040c73f9:::obj237:head) v3 finish latency 0.000010 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.108+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.108+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.108+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2180 con 0x3752000 2026-02-11T23:31:17.108+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:04510f17:::obj73:head)]) -- 0x3e03200 con 0x3752000 2026-02-11T23:31:17.108+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.109+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 64 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4cb7860 con 0x3752000 2026-02-11T23:31:17.109+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.109+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.109+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.109+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 65 ==== pg_backfill(progress 1.0 e 110/110 lb 1:04510f17:::obj73:head) ==== 1061+0+0 (unknown 0 0 0) 0x48ba580 con 0x3752000 2026-02-11T23:31:17.109+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.109+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:04510f17:::obj73:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.109+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:04510f17:::obj73:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.109+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.109+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.109+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 110/109 [PushOp(1:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.109+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.109+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.109+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:04510f17:::obj73:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.109+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.109+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.109+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0485ab12:::obj244:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.109+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0485ab12:::obj244:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.109+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0485ab12:::obj244:head 2026-02-11T23:31:17.109+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0485ab12:::obj244:head v 107'244 2026-02-11T23:31:17.109+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.109+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.109+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.109+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.109+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.109+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:04510f17:::obj73:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.109+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:04510f17:::obj73:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 110/110 lb 1:04510f17:::obj73:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.109+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.109+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:04510f17:::obj73:head) v3 2026-02-11T23:31:17.109+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:040c73f9:::obj237:head local-lis/les=109/110 n=13 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:04510f17:::obj73:head) 2026-02-11T23:31:17.109+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.109+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:04510f17:::obj73:head) v3 finish latency 0.000010 2026-02-11T23:31:17.109+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.109+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.109+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.109+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.109+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.109+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.109+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.109+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2600 con 0x3752000 2026-02-11T23:31:17.109+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0485ab12:::obj244:head)]) -- 0x3da3800 con 0x3752000 2026-02-11T23:31:17.109+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.110+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 66 ==== MOSDPGPush(1.0 110/109 [PushOp(1:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4cb7d40 con 0x3752000 2026-02-11T23:31:17.110+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.110+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.110+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 67 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0485ab12:::obj244:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bc100 con 0x3752000 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.110+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0485ab12:::obj244:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.110+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0485ab12:::obj244:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0485ab12:::obj244:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:04865490:::obj600:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:04865490:::obj600:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:04865490:::obj600:head 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:04865490:::obj600:head v 107'600 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.110+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0485ab12:::obj244:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.110+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0485ab12:::obj244:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:0485ab12:::obj244:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.110+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.110+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0485ab12:::obj244:head) v3 2026-02-11T23:31:17.110+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04510f17:::obj73:head local-lis/les=109/110 n=14 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0485ab12:::obj244:head) 2026-02-11T23:31:17.110+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.110+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0485ab12:::obj244:head) v3 finish latency 0.000011 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.110+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.110+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2300 con 0x3752000 2026-02-11T23:31:17.110+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:04865490:::obj600:head)]) -- 0x4af2780 con 0x3752000 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.110+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 68 ==== MOSDPGPush(1.0 110/109 [PushOp(1:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c18340 con 0x3752000 2026-02-11T23:31:17.110+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.110+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.110+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 69 ==== pg_backfill(progress 1.0 e 110/110 lb 1:04865490:::obj600:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bb600 con 0x3752000 2026-02-11T23:31:17.110+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:04865490:::obj600:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.110+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:04865490:::obj600:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:04865490:::obj600:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:048f8fd3:::obj518:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:048f8fd3:::obj518:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:048f8fd3:::obj518:head 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:048f8fd3:::obj518:head v 107'518 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.110+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.110+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.110+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:04865490:::obj600:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.110+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:04865490:::obj600:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 110/110 lb 1:04865490:::obj600:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.110+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.110+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:04865490:::obj600:head) v3 2026-02-11T23:31:17.110+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0485ab12:::obj244:head local-lis/les=109/110 n=15 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:04865490:::obj600:head) 2026-02-11T23:31:17.110+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.110+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:04865490:::obj600:head) v3 finish latency 0.000011 2026-02-11T23:31:17.110+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.111+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.111+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.111+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x3e03e00 con 0x3752000 2026-02-11T23:31:17.111+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:048f8fd3:::obj518:head)]) -- 0x4af2900 con 0x3752000 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.111+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 70 ==== MOSDPGPush(1.0 110/109 [PushOp(1:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c18820 con 0x3752000 2026-02-11T23:31:17.111+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.111+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.111+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.111+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 71 ==== pg_backfill(progress 1.0 e 110/110 lb 1:048f8fd3:::obj518:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bbb80 con 0x3752000 2026-02-11T23:31:17.111+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.111+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:048f8fd3:::obj518:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.111+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:048f8fd3:::obj518:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.111+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.111+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.111+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.111+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.111+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.111+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:048f8fd3:::obj518:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.111+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:04b71422:::obj226:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.111+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:04b71422:::obj226:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.111+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:04b71422:::obj226:head 2026-02-11T23:31:17.111+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:04b71422:::obj226:head v 107'226 2026-02-11T23:31:17.111+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.111+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.111+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.111+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:048f8fd3:::obj518:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.111+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:048f8fd3:::obj518:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 110/110 lb 1:048f8fd3:::obj518:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.111+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.111+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:048f8fd3:::obj518:head) v3 2026-02-11T23:31:17.111+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04865490:::obj600:head local-lis/les=109/110 n=16 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:048f8fd3:::obj518:head) 2026-02-11T23:31:17.111+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.111+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:048f8fd3:::obj518:head) v3 finish latency 0.000010 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.111+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.111+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.111+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4900 con 0x3752000 2026-02-11T23:31:17.111+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:04b71422:::obj226:head)]) -- 0x4af3500 con 0x3752000 2026-02-11T23:31:17.111+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.112+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 72 ==== MOSDPGPush(1.0 110/109 [PushOp(1:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c18d00 con 0x3752000 2026-02-11T23:31:17.112+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.112+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.112+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.112+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 73 ==== pg_backfill(progress 1.0 e 110/110 lb 1:04b71422:::obj226:head) ==== 1062+0+0 (unknown 0 0 0) 0x48ba000 con 0x3752000 2026-02-11T23:31:17.112+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:04b71422:::obj226:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.112+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.112+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:04b71422:::obj226:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.112+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.112+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.112+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.112+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.112+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.112+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:04b71422:::obj226:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.112+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.112+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.112+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:05651d4b:::obj461:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.112+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:05651d4b:::obj461:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.112+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:05651d4b:::obj461:head 2026-02-11T23:31:17.112+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:05651d4b:::obj461:head v 107'461 2026-02-11T23:31:17.112+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.112+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.112+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.112+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.112+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.112+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:04b71422:::obj226:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.112+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:04b71422:::obj226:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 110/110 lb 1:04b71422:::obj226:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.112+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.112+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:04b71422:::obj226:head) v3 2026-02-11T23:31:17.112+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:048f8fd3:::obj518:head local-lis/les=109/110 n=17 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:04b71422:::obj226:head) 2026-02-11T23:31:17.112+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.112+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:04b71422:::obj226:head) v3 finish latency 0.000010 2026-02-11T23:31:17.112+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.112+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.112+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.112+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.112+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.112+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.112+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.112+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4a80 con 0x3752000 2026-02-11T23:31:17.112+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:05651d4b:::obj461:head)]) -- 0x4af3680 con 0x3752000 2026-02-11T23:31:17.112+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.113+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 74 ==== MOSDPGPush(1.0 110/109 [PushOp(1:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c191e0 con 0x3752000 2026-02-11T23:31:17.113+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000017 epoch 110 2026-02-11T23:31:17.113+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.113+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 75 ==== pg_backfill(progress 1.0 e 110/110 lb 1:05651d4b:::obj461:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd180 con 0x3752000 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.113+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:05651d4b:::obj461:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.113+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05651d4b:::obj461:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 110/109 [PushOp(1:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05651d4b:::obj461:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:05793669:::obj415:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:05793669:::obj415:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:05793669:::obj415:head 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:05793669:::obj415:head v 107'415 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.113+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05651d4b:::obj461:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:05651d4b:::obj461:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:05651d4b:::obj461:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:05651d4b:::obj461:head) v3 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:04b71422:::obj226:head local-lis/les=109/110 n=18 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:05651d4b:::obj461:head) 2026-02-11T23:31:17.113+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:05651d4b:::obj461:head) v3 finish latency 0.000010 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.113+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4c00 con 0x3752000 2026-02-11T23:31:17.113+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:05793669:::obj415:head)]) -- 0x4cb5200 con 0x3752000 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.113+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 76 ==== MOSDPGPush(1.0 110/109 [PushOp(1:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c196c0 con 0x3752000 2026-02-11T23:31:17.113+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.113+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.113+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 77 ==== pg_backfill(progress 1.0 e 110/110 lb 1:05793669:::obj415:head) ==== 1062+0+0 (unknown 0 0 0) 0x46e3600 con 0x3752000 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.113+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:05793669:::obj415:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.113+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05793669:::obj415:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05793669:::obj415:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:05e4a07c:::obj265:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:05e4a07c:::obj265:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:05e4a07c:::obj265:head 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:05e4a07c:::obj265:head v 107'265 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.113+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.113+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05793669:::obj415:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:05793669:::obj415:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:05793669:::obj415:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:05793669:::obj415:head) v3 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05651d4b:::obj461:head local-lis/les=109/110 n=19 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:05793669:::obj415:head) 2026-02-11T23:31:17.113+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:05793669:::obj415:head) v3 finish latency 0.000009 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.113+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.113+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.113+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4d80 con 0x3752000 2026-02-11T23:31:17.113+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:05e4a07c:::obj265:head)]) -- 0x4cb5500 con 0x3752000 2026-02-11T23:31:17.113+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.114+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 78 ==== MOSDPGPush(1.0 110/109 [PushOp(1:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c19ba0 con 0x3752000 2026-02-11T23:31:17.114+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.114+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.114+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.114+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 79 ==== pg_backfill(progress 1.0 e 110/110 lb 1:05e4a07c:::obj265:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd700 con 0x3752000 2026-02-11T23:31:17.114+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.114+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:05e4a07c:::obj265:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.114+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05e4a07c:::obj265:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.114+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.114+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.114+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.114+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.114+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05e4a07c:::obj265:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.114+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.114+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.114+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.114+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:05eb3c5d:::obj514:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.114+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:05eb3c5d:::obj514:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.114+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:05eb3c5d:::obj514:head 2026-02-11T23:31:17.114+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:05eb3c5d:::obj514:head v 107'514 2026-02-11T23:31:17.114+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.114+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.114+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.114+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.114+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.114+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05e4a07c:::obj265:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.114+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:05e4a07c:::obj265:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:05e4a07c:::obj265:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.114+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.114+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:05e4a07c:::obj265:head) v3 2026-02-11T23:31:17.114+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05793669:::obj415:head local-lis/les=109/110 n=20 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:05e4a07c:::obj265:head) 2026-02-11T23:31:17.114+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.114+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:05e4a07c:::obj265:head) v3 finish latency 0.000010 2026-02-11T23:31:17.114+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.114+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.114+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.114+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.114+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.114+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.114+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.114+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4f00 con 0x3752000 2026-02-11T23:31:17.114+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:05eb3c5d:::obj514:head)]) -- 0x4cb5800 con 0x3752000 2026-02-11T23:31:17.114+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.114+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 80 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bb81a0 con 0x3752000 2026-02-11T23:31:17.114+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.114+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.114+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.114+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 81 ==== pg_backfill(progress 1.0 e 110/110 lb 1:05eb3c5d:::obj514:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c1c000 con 0x3752000 2026-02-11T23:31:17.114+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.114+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:05eb3c5d:::obj514:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.114+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.114+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05eb3c5d:::obj514:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.114+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05eb3c5d:::obj514:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0698d46e:::obj173:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.115+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0698d46e:::obj173:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0698d46e:::obj173:head 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0698d46e:::obj173:head v 107'173 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.115+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.115+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:05eb3c5d:::obj514:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.115+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:05eb3c5d:::obj514:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 110/110 lb 1:05eb3c5d:::obj514:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.115+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.115+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:05eb3c5d:::obj514:head) v3 2026-02-11T23:31:17.115+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05e4a07c:::obj265:head local-lis/les=109/110 n=21 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:05eb3c5d:::obj514:head) 2026-02-11T23:31:17.115+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.115+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:05eb3c5d:::obj514:head) v3 finish latency 0.000011 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.115+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.115+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.115+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb5080 con 0x3752000 2026-02-11T23:31:17.115+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0698d46e:::obj173:head)]) -- 0x4cb5b00 con 0x3752000 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.115+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 82 ==== MOSDPGPush(1.0 110/109 [PushOp(1:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bb8680 con 0x3752000 2026-02-11T23:31:17.115+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 110 2026-02-11T23:31:17.115+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.115+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.115+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 83 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0698d46e:::obj173:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c1c580 con 0x3752000 2026-02-11T23:31:17.115+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.115+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0698d46e:::obj173:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.115+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0698d46e:::obj173:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.115+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.115+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0698d46e:::obj173:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:06c5d68c:::obj504:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.115+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:06c5d68c:::obj504:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:06c5d68c:::obj504:head 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:06c5d68c:::obj504:head v 107'504 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.115+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.115+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.115+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0698d46e:::obj173:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.115+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0698d46e:::obj173:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 110/110 lb 1:0698d46e:::obj173:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.115+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.115+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0698d46e:::obj173:head) v3 2026-02-11T23:31:17.115+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:05eb3c5d:::obj514:head local-lis/les=109/110 n=22 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0698d46e:::obj173:head) 2026-02-11T23:31:17.115+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.115+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0698d46e:::obj173:head) v3 finish latency 0.000011 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.115+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.116+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.116+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.116+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af3800 con 0x3752000 2026-02-11T23:31:17.116+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:06c5d68c:::obj504:head)]) -- 0x4cb5e00 con 0x3752000 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.116+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 84 ==== MOSDPGPush(1.0 110/109 [PushOp(1:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bb8b60 con 0x3752000 2026-02-11T23:31:17.116+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.116+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.116+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.116+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 85 ==== pg_backfill(progress 1.0 e 110/110 lb 1:06c5d68c:::obj504:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c1cb00 con 0x3752000 2026-02-11T23:31:17.116+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:06c5d68c:::obj504:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.116+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.116+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:06c5d68c:::obj504:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.116+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.116+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.116+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.116+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:06c5d68c:::obj504:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.116+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.116+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.116+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:06d56074:::obj162:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.116+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:06d56074:::obj162:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.116+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:06d56074:::obj162:head 2026-02-11T23:31:17.116+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:06d56074:::obj162:head v 107'162 2026-02-11T23:31:17.116+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.116+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.116+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.116+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:06c5d68c:::obj504:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.116+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:06c5d68c:::obj504:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 110/110 lb 1:06c5d68c:::obj504:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.116+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.116+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:06c5d68c:::obj504:head) v3 2026-02-11T23:31:17.116+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0698d46e:::obj173:head local-lis/les=109/110 n=23 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:06c5d68c:::obj504:head) 2026-02-11T23:31:17.116+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.116+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:06c5d68c:::obj504:head) v3 finish latency 0.000010 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.116+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.116+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.116+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4000 con 0x3752000 2026-02-11T23:31:17.116+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:06d56074:::obj162:head)]) -- 0x4af3b00 con 0x3752000 2026-02-11T23:31:17.116+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.117+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 86 ==== MOSDPGPush(1.0 110/109 [PushOp(1:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bb9040 con 0x3752000 2026-02-11T23:31:17.117+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 110 2026-02-11T23:31:17.117+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.117+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 87 ==== pg_backfill(progress 1.0 e 110/110 lb 1:06d56074:::obj162:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c1d080 con 0x3752000 2026-02-11T23:31:17.117+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:06d56074:::obj162:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.117+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:06d56074:::obj162:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 110/109 [PushOp(1:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:06d56074:::obj162:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:07c44a4d:::obj181:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:07c44a4d:::obj181:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:07c44a4d:::obj181:head 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:07c44a4d:::obj181:head v 107'181 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.117+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:06d56074:::obj162:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.117+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:06d56074:::obj162:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 110/110 lb 1:06d56074:::obj162:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.117+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.117+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:06d56074:::obj162:head) v3 2026-02-11T23:31:17.117+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06c5d68c:::obj504:head local-lis/les=109/110 n=24 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:06d56074:::obj162:head) 2026-02-11T23:31:17.117+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.117+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:06d56074:::obj162:head) v3 finish latency 0.000010 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.117+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.117+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4300 con 0x3752000 2026-02-11T23:31:17.117+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:07c44a4d:::obj181:head)]) -- 0x4af2480 con 0x3752000 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.117+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 88 ==== MOSDPGPush(1.0 110/109 [PushOp(1:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bb9520 con 0x3752000 2026-02-11T23:31:17.117+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.117+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.117+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 89 ==== pg_backfill(progress 1.0 e 110/110 lb 1:07c44a4d:::obj181:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c1d600 con 0x3752000 2026-02-11T23:31:17.117+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:07c44a4d:::obj181:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.117+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:07c44a4d:::obj181:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:07c44a4d:::obj181:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 110/109 [PushOp(1:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:080cd0cf:::obj146:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:080cd0cf:::obj146:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:080cd0cf:::obj146:head 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:080cd0cf:::obj146:head v 107'146 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.117+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.117+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.117+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:07c44a4d:::obj181:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.117+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:07c44a4d:::obj181:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 110/110 lb 1:07c44a4d:::obj181:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.117+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.117+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:07c44a4d:::obj181:head) v3 2026-02-11T23:31:17.117+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:06d56074:::obj162:head local-lis/les=109/110 n=25 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:07c44a4d:::obj181:head) 2026-02-11T23:31:17.117+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.117+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:07c44a4d:::obj181:head) v3 finish latency 0.000010 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.117+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.118+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.118+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.118+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2600 con 0x3752000 2026-02-11T23:31:17.118+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:080cd0cf:::obj146:head)]) -- 0x3e02c00 con 0x3752000 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.118+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 90 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bb9a00 con 0x3752000 2026-02-11T23:31:17.118+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.118+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.118+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.118+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 91 ==== pg_backfill(progress 1.0 e 110/110 lb 1:080cd0cf:::obj146:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c1db80 con 0x3752000 2026-02-11T23:31:17.118+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.118+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:080cd0cf:::obj146:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.118+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:080cd0cf:::obj146:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.118+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.118+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.118+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:080cd0cf:::obj146:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.118+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.118+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.118+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.118+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0877fdce:::obj530:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.118+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0877fdce:::obj530:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.118+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0877fdce:::obj530:head 2026-02-11T23:31:17.118+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0877fdce:::obj530:head v 107'530 2026-02-11T23:31:17.118+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.118+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.118+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.118+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:080cd0cf:::obj146:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.118+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:080cd0cf:::obj146:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 110/110 lb 1:080cd0cf:::obj146:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.118+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.118+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:080cd0cf:::obj146:head) v3 2026-02-11T23:31:17.118+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:07c44a4d:::obj181:head local-lis/les=109/110 n=26 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:080cd0cf:::obj146:head) 2026-02-11T23:31:17.118+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.118+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:080cd0cf:::obj146:head) v3 finish latency 0.000010 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.118+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.118+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.118+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2300 con 0x3752000 2026-02-11T23:31:17.118+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0877fdce:::obj530:head)]) -- 0x3e03200 con 0x3752000 2026-02-11T23:31:17.118+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.119+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 92 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ce8000 con 0x3752000 2026-02-11T23:31:17.119+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.119+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.119+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 93 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0877fdce:::obj530:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c1e100 con 0x3752000 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.119+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0877fdce:::obj530:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.119+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0877fdce:::obj530:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0877fdce:::obj530:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0881dee6:::obj412:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0881dee6:::obj412:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0881dee6:::obj412:head 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0881dee6:::obj412:head v 107'412 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.119+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0877fdce:::obj530:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.119+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0877fdce:::obj530:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 110/110 lb 1:0877fdce:::obj530:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.119+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.119+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0877fdce:::obj530:head) v3 2026-02-11T23:31:17.119+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:080cd0cf:::obj146:head local-lis/les=109/110 n=27 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0877fdce:::obj530:head) 2026-02-11T23:31:17.119+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.119+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0877fdce:::obj530:head) v3 finish latency 0.000009 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.119+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.119+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2780 con 0x3752000 2026-02-11T23:31:17.119+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0881dee6:::obj412:head)]) -- 0x3da3800 con 0x3752000 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.119+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 94 ==== MOSDPGPush(1.0 110/109 [PushOp(1:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4ce84e0 con 0x3752000 2026-02-11T23:31:17.119+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.119+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.119+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 95 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0881dee6:::obj412:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c1e680 con 0x3752000 2026-02-11T23:31:17.119+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0881dee6:::obj412:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.119+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0881dee6:::obj412:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0881dee6:::obj412:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:08a5234b:::obj62:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:08a5234b:::obj62:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:08a5234b:::obj62:head 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:08a5234b:::obj62:head v 107'62 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.119+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.119+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.119+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.119+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0881dee6:::obj412:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.119+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0881dee6:::obj412:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 110/110 lb 1:0881dee6:::obj412:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.119+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.119+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0881dee6:::obj412:head) v3 2026-02-11T23:31:17.119+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0877fdce:::obj530:head local-lis/les=109/110 n=28 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0881dee6:::obj412:head) 2026-02-11T23:31:17.120+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.120+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0881dee6:::obj412:head) v3 finish latency 0.000013 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.120+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.120+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.120+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2900 con 0x3752000 2026-02-11T23:31:17.120+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:08a5234b:::obj62:head)]) -- 0x4cb4900 con 0x3752000 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.120+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 96 ==== MOSDPGPush(1.0 110/109 [PushOp(1:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ce89c0 con 0x3752000 2026-02-11T23:31:17.120+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.120+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.120+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.120+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 97 ==== pg_backfill(progress 1.0 e 110/110 lb 1:08a5234b:::obj62:head) ==== 1061+0+0 (unknown 0 0 0) 0x4c1ec00 con 0x3752000 2026-02-11T23:31:17.120+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.120+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:08a5234b:::obj62:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.120+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:08a5234b:::obj62:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.120+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.120+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.120+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.120+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.120+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:08a5234b:::obj62:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.120+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.120+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:08fb0692:::obj136:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.120+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:08fb0692:::obj136:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.120+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:08fb0692:::obj136:head 2026-02-11T23:31:17.120+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:08fb0692:::obj136:head v 107'136 2026-02-11T23:31:17.120+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.120+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.120+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.120+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:08a5234b:::obj62:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.120+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:08a5234b:::obj62:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 110/110 lb 1:08a5234b:::obj62:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.120+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.120+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:08a5234b:::obj62:head) v3 2026-02-11T23:31:17.120+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0881dee6:::obj412:head local-lis/les=109/110 n=29 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:08a5234b:::obj62:head) 2026-02-11T23:31:17.120+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.120+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:08a5234b:::obj62:head) v3 finish latency 0.000010 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.120+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.120+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.120+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af3500 con 0x3752000 2026-02-11T23:31:17.120+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:08fb0692:::obj136:head)]) -- 0x4cb4a80 con 0x3752000 2026-02-11T23:31:17.120+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.121+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 98 ==== MOSDPGPush(1.0 110/109 [PushOp(1:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4ce8ea0 con 0x3752000 2026-02-11T23:31:17.121+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 110 2026-02-11T23:31:17.121+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.121+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.121+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 99 ==== pg_backfill(progress 1.0 e 110/110 lb 1:08fb0692:::obj136:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c1f180 con 0x3752000 2026-02-11T23:31:17.121+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.121+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:08fb0692:::obj136:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.121+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:08fb0692:::obj136:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.121+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.121+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.121+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 110/109 [PushOp(1:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.121+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.121+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.121+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:08fb0692:::obj136:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.121+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.121+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.121+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:095c6b7a:::obj79:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.121+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:095c6b7a:::obj79:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.121+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:095c6b7a:::obj79:head 2026-02-11T23:31:17.121+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:095c6b7a:::obj79:head v 107'79 2026-02-11T23:31:17.121+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.121+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.121+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.121+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.121+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.121+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:08fb0692:::obj136:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.121+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:08fb0692:::obj136:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 110/110 lb 1:08fb0692:::obj136:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.121+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.121+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:08fb0692:::obj136:head) v3 2026-02-11T23:31:17.121+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08a5234b:::obj62:head local-lis/les=109/110 n=30 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:08fb0692:::obj136:head) 2026-02-11T23:31:17.121+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.121+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:08fb0692:::obj136:head) v3 finish latency 0.000010 2026-02-11T23:31:17.121+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.121+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.121+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.121+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.121+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.121+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.121+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.121+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af3680 con 0x3752000 2026-02-11T23:31:17.121+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:095c6b7a:::obj79:head)]) -- 0x4cb5200 con 0x3752000 2026-02-11T23:31:17.121+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.121+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 100 ==== MOSDPGPush(1.0 110/109 [PushOp(1:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ce9380 con 0x3752000 2026-02-11T23:31:17.121+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 110 2026-02-11T23:31:17.121+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.121+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.121+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.121+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 101 ==== pg_backfill(progress 1.0 e 110/110 lb 1:095c6b7a:::obj79:head) ==== 1061+0+0 (unknown 0 0 0) 0x4c1f700 con 0x3752000 2026-02-11T23:31:17.122+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:095c6b7a:::obj79:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.122+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:095c6b7a:::obj79:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 110/109 [PushOp(1:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:095c6b7a:::obj79:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:097a0595:::obj169:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:097a0595:::obj169:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:097a0595:::obj169:head 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:097a0595:::obj169:head v 107'169 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.122+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:095c6b7a:::obj79:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:095c6b7a:::obj79:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 110/110 lb 1:095c6b7a:::obj79:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:095c6b7a:::obj79:head) v3 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:08fb0692:::obj136:head local-lis/les=109/110 n=31 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:095c6b7a:::obj79:head) 2026-02-11T23:31:17.122+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:095c6b7a:::obj79:head) v3 finish latency 0.000010 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.122+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4d80 con 0x3752000 2026-02-11T23:31:17.122+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:097a0595:::obj169:head)]) -- 0x4af3800 con 0x3752000 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.122+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 102 ==== MOSDPGPush(1.0 110/109 [PushOp(1:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ce9860 con 0x3752000 2026-02-11T23:31:17.122+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.122+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.122+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 103 ==== pg_backfill(progress 1.0 e 110/110 lb 1:097a0595:::obj169:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c94000 con 0x3752000 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.122+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:097a0595:::obj169:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.122+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:097a0595:::obj169:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:097a0595:::obj169:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:09d0cce6:::obj481:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:09d0cce6:::obj481:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:09d0cce6:::obj481:head 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:09d0cce6:::obj481:head v 107'481 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.122+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.122+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:097a0595:::obj169:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:097a0595:::obj169:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:097a0595:::obj169:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:097a0595:::obj169:head) v3 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:095c6b7a:::obj79:head local-lis/les=109/110 n=32 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:097a0595:::obj169:head) 2026-02-11T23:31:17.122+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:097a0595:::obj169:head) v3 finish latency 0.000010 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.122+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.122+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.122+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb5500 con 0x3752000 2026-02-11T23:31:17.122+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:09d0cce6:::obj481:head)]) -- 0x4af2480 con 0x3752000 2026-02-11T23:31:17.122+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.123+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 104 ==== MOSDPGPush(1.0 110/109 [PushOp(1:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ce9d40 con 0x3752000 2026-02-11T23:31:17.123+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.123+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.123+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.123+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 105 ==== pg_backfill(progress 1.0 e 110/110 lb 1:09d0cce6:::obj481:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c94580 con 0x3752000 2026-02-11T23:31:17.123+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:09d0cce6:::obj481:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.123+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:09d0cce6:::obj481:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.123+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.123+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.123+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.123+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.123+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.123+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.123+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:09d0cce6:::obj481:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.123+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.123+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.123+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:09eb3833:::obj488:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.123+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:09eb3833:::obj488:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.123+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:09eb3833:::obj488:head 2026-02-11T23:31:17.123+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:09eb3833:::obj488:head v 107'488 2026-02-11T23:31:17.123+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.123+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.123+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:17.123+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.123+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.123+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.123+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:09d0cce6:::obj481:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.123+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:09d0cce6:::obj481:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 1.0 e 110/110 lb 1:09d0cce6:::obj481:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.123+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.123+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:09d0cce6:::obj481:head) v3 2026-02-11T23:31:17.123+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:097a0595:::obj169:head local-lis/les=109/110 n=33 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:09d0cce6:::obj481:head) 2026-02-11T23:31:17.123+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.123+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:09d0cce6:::obj481:head) v3 finish latency 0.000023 2026-02-11T23:31:17.123+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.123+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.123+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.123+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.123+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.123+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.123+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb5080 con 0x3752000 2026-02-11T23:31:17.123+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:09eb3833:::obj488:head)]) -- 0x4cb4f00 con 0x3752000 2026-02-11T23:31:17.123+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.124+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 106 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c98340 con 0x3752000 2026-02-11T23:31:17.124+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.124+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.124+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 107 ==== pg_backfill(progress 1.0 e 110/110 lb 1:09eb3833:::obj488:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c94b00 con 0x3752000 2026-02-11T23:31:17.124+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:09eb3833:::obj488:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.124+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:09eb3833:::obj488:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.124+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.124+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.124+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.124+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:09eb3833:::obj488:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.124+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.124+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.124+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0a4fcbe7:::obj507:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0a4fcbe7:::obj507:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.124+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0a4fcbe7:::obj507:head 2026-02-11T23:31:17.124+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0a4fcbe7:::obj507:head v 107'507 2026-02-11T23:31:17.124+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.124+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.124+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.124+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.124+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:09eb3833:::obj488:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.124+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:09eb3833:::obj488:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 110/110 lb 1:09eb3833:::obj488:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.124+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.124+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:09eb3833:::obj488:head) v3 2026-02-11T23:31:17.124+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09d0cce6:::obj481:head local-lis/les=109/110 n=34 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:09eb3833:::obj488:head) 2026-02-11T23:31:17.124+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.124+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:09eb3833:::obj488:head) v3 finish latency 0.000015 2026-02-11T23:31:17.124+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.124+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.124+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.124+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4000 con 0x3752000 2026-02-11T23:31:17.124+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0a4fcbe7:::obj507:head)]) -- 0x4cb5b00 con 0x3752000 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.124+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 108 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c98820 con 0x3752000 2026-02-11T23:31:17.124+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 110 2026-02-11T23:31:17.124+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.124+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.124+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 109 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0a4fcbe7:::obj507:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c95080 con 0x3752000 2026-02-11T23:31:17.124+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0a4fcbe7:::obj507:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.124+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.124+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0a4fcbe7:::obj507:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.124+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.124+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.124+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 110/109 [PushOp(1:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.124+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.124+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.124+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.124+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0a4fcbe7:::obj507:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.124+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0b3f05c8:::obj589:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.124+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0b3f05c8:::obj589:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.124+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0b3f05c8:::obj589:head 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0b3f05c8:::obj589:head v 107'589 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.125+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.125+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0a4fcbe7:::obj507:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0a4fcbe7:::obj507:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 110/110 lb 1:0a4fcbe7:::obj507:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0a4fcbe7:::obj507:head) v3 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:09eb3833:::obj488:head local-lis/les=109/110 n=35 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0a4fcbe7:::obj507:head) 2026-02-11T23:31:17.125+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0a4fcbe7:::obj507:head) v3 finish latency 0.000019 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.125+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.125+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4300 con 0x3752000 2026-02-11T23:31:17.125+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0b3f05c8:::obj589:head)]) -- 0x4af2300 con 0x3752000 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.125+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 110 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c98d00 con 0x3752000 2026-02-11T23:31:17.125+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.125+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.125+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.125+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 111 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0b3f05c8:::obj589:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c95600 con 0x3752000 2026-02-11T23:31:17.125+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0b3f05c8:::obj589:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.125+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.125+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0b3f05c8:::obj589:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.125+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.125+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0b3f05c8:::obj589:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0bdade2e:::obj246:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.125+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0bdade2e:::obj246:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0bdade2e:::obj246:head 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0bdade2e:::obj246:head v 107'246 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.125+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.125+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.125+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0b3f05c8:::obj589:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0b3f05c8:::obj589:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 110/110 lb 1:0b3f05c8:::obj589:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0b3f05c8:::obj589:head) v3 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0a4fcbe7:::obj507:head local-lis/les=109/110 n=36 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0b3f05c8:::obj589:head) 2026-02-11T23:31:17.125+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0b3f05c8:::obj589:head) v3 finish latency 0.000013 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.125+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.125+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.125+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4900 con 0x3752000 2026-02-11T23:31:17.125+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0bdade2e:::obj246:head)]) -- 0x4af2900 con 0x3752000 2026-02-11T23:31:17.125+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.126+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 112 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c991e0 con 0x3752000 2026-02-11T23:31:17.126+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.126+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.126+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.126+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.126+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 113 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0bdade2e:::obj246:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c95b80 con 0x3752000 2026-02-11T23:31:17.126+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.126+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0bdade2e:::obj246:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 110 2026-02-11T23:31:17.126+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.126+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0bdade2e:::obj246:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.126+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.126+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.126+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.126+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0bdade2e:::obj246:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.126+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.126+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.126+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0bde2796:::obj413:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.126+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0bde2796:::obj413:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.126+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0bde2796:::obj413:head 2026-02-11T23:31:17.126+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0bde2796:::obj413:head v 107'413 2026-02-11T23:31:17.126+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.126+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.126+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.126+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.126+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.126+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0bdade2e:::obj246:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.126+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0bdade2e:::obj246:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:0bdade2e:::obj246:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.126+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.126+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0bdade2e:::obj246:head) v3 2026-02-11T23:31:17.126+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0b3f05c8:::obj589:head local-lis/les=109/110 n=37 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0bdade2e:::obj246:head) 2026-02-11T23:31:17.126+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.126+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0bdade2e:::obj246:head) v3 finish latency 0.000018 2026-02-11T23:31:17.126+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.126+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.126+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.126+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.126+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.126+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.126+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.126+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4a80 con 0x3752000 2026-02-11T23:31:17.126+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0bde2796:::obj413:head)]) -- 0x4af3680 con 0x3752000 2026-02-11T23:31:17.126+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.127+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 114 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c996c0 con 0x3752000 2026-02-11T23:31:17.127+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 110 2026-02-11T23:31:17.127+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.127+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 115 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0bde2796:::obj413:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c96100 con 0x3752000 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.127+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0bde2796:::obj413:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.127+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0bde2796:::obj413:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0bde2796:::obj413:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0c03281d:::obj596:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0c03281d:::obj596:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0c03281d:::obj596:head 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0c03281d:::obj596:head v 107'596 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.127+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0bde2796:::obj413:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.127+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0bde2796:::obj413:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 110/110 lb 1:0bde2796:::obj413:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.127+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.127+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0bde2796:::obj413:head) v3 2026-02-11T23:31:17.127+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bdade2e:::obj246:head local-lis/les=109/110 n=38 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0bde2796:::obj413:head) 2026-02-11T23:31:17.127+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.127+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0bde2796:::obj413:head) v3 finish latency 0.000017 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.127+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.127+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb5200 con 0x3752000 2026-02-11T23:31:17.127+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0c03281d:::obj596:head)]) -- 0x3e02c00 con 0x3752000 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.127+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 116 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c99ba0 con 0x3752000 2026-02-11T23:31:17.127+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.127+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.127+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 117 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0c03281d:::obj596:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c96680 con 0x3752000 2026-02-11T23:31:17.127+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0c03281d:::obj596:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.127+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0c03281d:::obj596:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0c03281d:::obj596:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0cfeb9e4:::obj357:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0cfeb9e4:::obj357:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0cfeb9e4:::obj357:head 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0cfeb9e4:::obj357:head v 107'357 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.127+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.127+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.127+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0c03281d:::obj596:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.127+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0c03281d:::obj596:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:0c03281d:::obj596:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.127+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.127+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0c03281d:::obj596:head) v3 2026-02-11T23:31:17.127+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0bde2796:::obj413:head local-lis/les=109/110 n=39 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0c03281d:::obj596:head) 2026-02-11T23:31:17.127+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.127+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0c03281d:::obj596:head) v3 finish latency 0.000019 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.127+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.128+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.128+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.128+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23380 con 0x3752000 2026-02-11T23:31:17.128+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0cfeb9e4:::obj357:head)]) -- 0x4c22000 con 0x3752000 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.128+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 118 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4cae1a0 con 0x3752000 2026-02-11T23:31:17.128+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.128+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.128+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.128+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 119 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0cfeb9e4:::obj357:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c96c00 con 0x3752000 2026-02-11T23:31:17.128+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.128+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0cfeb9e4:::obj357:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.128+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0cfeb9e4:::obj357:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.128+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.128+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.128+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.128+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.128+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0cfeb9e4:::obj357:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.128+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.128+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0dc4a75f:::obj282:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.128+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0dc4a75f:::obj282:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.128+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0dc4a75f:::obj282:head 2026-02-11T23:31:17.128+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0dc4a75f:::obj282:head v 107'282 2026-02-11T23:31:17.128+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.128+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.128+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.128+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0cfeb9e4:::obj357:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.128+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0cfeb9e4:::obj357:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 110/110 lb 1:0cfeb9e4:::obj357:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.128+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.128+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0cfeb9e4:::obj357:head) v3 2026-02-11T23:31:17.128+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0c03281d:::obj596:head local-lis/les=109/110 n=40 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0cfeb9e4:::obj357:head) 2026-02-11T23:31:17.128+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.128+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0cfeb9e4:::obj357:head) v3 finish latency 0.000013 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.128+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.128+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.128+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23500 con 0x3752000 2026-02-11T23:31:17.128+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0dc4a75f:::obj282:head)]) -- 0x4c22300 con 0x3752000 2026-02-11T23:31:17.128+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.129+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 120 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4cae680 con 0x3752000 2026-02-11T23:31:17.129+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 110 2026-02-11T23:31:17.129+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.129+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 121 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0dc4a75f:::obj282:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c97180 con 0x3752000 2026-02-11T23:31:17.129+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0dc4a75f:::obj282:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 110 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 110/109 [PushOp(1:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.129+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0dc4a75f:::obj282:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0dc4a75f:::obj282:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0e39285d:::obj525:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0e39285d:::obj525:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0e39285d:::obj525:head 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0e39285d:::obj525:head v 107'525 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.129+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0dc4a75f:::obj282:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0dc4a75f:::obj282:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 110/110 lb 1:0dc4a75f:::obj282:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0dc4a75f:::obj282:head) v3 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0cfeb9e4:::obj357:head local-lis/les=109/110 n=41 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0dc4a75f:::obj282:head) 2026-02-11T23:31:17.129+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0dc4a75f:::obj282:head) v3 finish latency 0.000013 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.129+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23680 con 0x3752000 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.129+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0e39285d:::obj525:head)]) -- 0x4c22600 con 0x3752000 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.129+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 122 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4caeb60 con 0x3752000 2026-02-11T23:31:17.129+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 110 2026-02-11T23:31:17.129+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.129+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 123 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0e39285d:::obj525:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c97700 con 0x3752000 2026-02-11T23:31:17.129+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e39285d:::obj525:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.129+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e39285d:::obj525:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 110/109 [PushOp(1:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e39285d:::obj525:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0e3dc695:::obj78:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0e3dc695:::obj78:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0e3dc695:::obj78:head 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0e3dc695:::obj78:head v 107'78 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.129+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e39285d:::obj525:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e39285d:::obj525:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 110/110 lb 1:0e39285d:::obj525:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0e39285d:::obj525:head) v3 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0dc4a75f:::obj282:head local-lis/les=109/110 n=42 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0e39285d:::obj525:head) 2026-02-11T23:31:17.129+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.129+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e39285d:::obj525:head) v3 finish latency 0.000016 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.129+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.129+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.129+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23980 con 0x3752000 2026-02-11T23:31:17.129+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0e3dc695:::obj78:head)]) -- 0x4c23800 con 0x3752000 2026-02-11T23:31:17.129+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.130+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 124 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4caf040 con 0x3752000 2026-02-11T23:31:17.130+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.130+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.130+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.130+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.130+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 125 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0e3dc695:::obj78:head) ==== 1061+0+0 (unknown 0 0 0) 0x4c32000 con 0x3752000 2026-02-11T23:31:17.130+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e3dc695:::obj78:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.130+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.130+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e3dc695:::obj78:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.130+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.130+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.130+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.130+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.130+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.130+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.130+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e3dc695:::obj78:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.130+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.130+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0e5b0695:::obj117:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.130+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0e5b0695:::obj117:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.130+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0e5b0695:::obj117:head 2026-02-11T23:31:17.130+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0e5b0695:::obj117:head v 107'117 2026-02-11T23:31:17.130+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.130+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.130+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.130+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.130+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.130+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.130+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e3dc695:::obj78:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.130+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e3dc695:::obj78:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 110/110 lb 1:0e3dc695:::obj78:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.130+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.130+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0e3dc695:::obj78:head) v3 2026-02-11T23:31:17.130+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e39285d:::obj525:head local-lis/les=109/110 n=43 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0e3dc695:::obj78:head) 2026-02-11T23:31:17.130+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.130+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e3dc695:::obj78:head) v3 finish latency 0.000015 2026-02-11T23:31:17.130+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.130+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.130+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.130+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.130+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.130+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.130+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23e00 con 0x3752000 2026-02-11T23:31:17.130+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0e5b0695:::obj117:head)]) -- 0x4c23b00 con 0x3752000 2026-02-11T23:31:17.130+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.131+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 126 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4caf520 con 0x3752000 2026-02-11T23:31:17.131+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.131+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.131+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 127 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0e5b0695:::obj117:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c32580 con 0x3752000 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.131+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e5b0695:::obj117:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.131+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e5b0695:::obj117:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 110/109 [PushOp(1:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e5b0695:::obj117:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0e5e5c87:::obj521:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0e5e5c87:::obj521:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0e5e5c87:::obj521:head 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0e5e5c87:::obj521:head v 107'521 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.131+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e5b0695:::obj117:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e5b0695:::obj117:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 110/110 lb 1:0e5b0695:::obj117:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0e5b0695:::obj117:head) v3 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e3dc695:::obj78:head local-lis/les=109/110 n=44 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0e5b0695:::obj117:head) 2026-02-11T23:31:17.131+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e5b0695:::obj117:head) v3 finish latency 0.000014 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.131+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4d80 con 0x3752000 2026-02-11T23:31:17.131+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0e5e5c87:::obj521:head)]) -- 0x4c22900 con 0x3752000 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.131+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 128 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4cafa00 con 0x3752000 2026-02-11T23:31:17.131+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.131+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.131+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 129 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0e5e5c87:::obj521:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c32b00 con 0x3752000 2026-02-11T23:31:17.131+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e5e5c87:::obj521:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.131+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e5e5c87:::obj521:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 110/109 [PushOp(1:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e5e5c87:::obj521:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0ed26813:::obj311:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0ed26813:::obj311:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0ed26813:::obj311:head 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0ed26813:::obj311:head v 107'311 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.131+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.131+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0e5e5c87:::obj521:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e5e5c87:::obj521:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 110/110 lb 1:0e5e5c87:::obj521:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0e5e5c87:::obj521:head) v3 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5b0695:::obj117:head local-lis/les=109/110 n=45 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0e5e5c87:::obj521:head) 2026-02-11T23:31:17.131+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0e5e5c87:::obj521:head) v3 finish latency 0.000011 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.131+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.131+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb5500 con 0x3752000 2026-02-11T23:31:17.131+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0ed26813:::obj311:head)]) -- 0x4c22c00 con 0x3752000 2026-02-11T23:31:17.131+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.131+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.132+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 130 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c3e000 con 0x3752000 2026-02-11T23:31:17.132+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.132+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.132+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.132+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 131 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0ed26813:::obj311:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c33080 con 0x3752000 2026-02-11T23:31:17.132+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0ed26813:::obj311:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.132+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.132+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0ed26813:::obj311:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.132+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.132+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.132+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.132+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.132+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.132+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.132+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0ed26813:::obj311:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.132+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.132+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0ed3c684:::obj270:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.132+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0ed3c684:::obj270:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.132+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0ed3c684:::obj270:head 2026-02-11T23:31:17.132+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0ed3c684:::obj270:head v 107'270 2026-02-11T23:31:17.132+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.132+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.132+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.132+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.132+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.132+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0ed26813:::obj311:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.132+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0ed26813:::obj311:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:0ed26813:::obj311:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.132+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.132+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0ed26813:::obj311:head) v3 2026-02-11T23:31:17.132+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0e5e5c87:::obj521:head local-lis/les=109/110 n=46 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0ed26813:::obj311:head) 2026-02-11T23:31:17.132+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.132+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0ed26813:::obj311:head) v3 finish latency 0.000014 2026-02-11T23:31:17.132+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.132+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.132+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.132+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.132+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.132+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.132+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.132+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb5080 con 0x3752000 2026-02-11T23:31:17.132+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0ed3c684:::obj270:head)]) -- 0x4c22f00 con 0x3752000 2026-02-11T23:31:17.132+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.132+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 132 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x4c3e4e0 con 0x3752000 2026-02-11T23:31:17.133+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.133+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.133+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 133 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0ed3c684:::obj270:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c33600 con 0x3752000 2026-02-11T23:31:17.133+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0ed3c684:::obj270:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.133+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0ed3c684:::obj270:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 110/109 [PushOp(1:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0ed3c684:::obj270:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0ee9ae15:::obj4:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0ee9ae15:::obj4:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0ee9ae15:::obj4:head 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0ee9ae15:::obj4:head v 107'4 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.133+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0ed3c684:::obj270:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0ed3c684:::obj270:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 110/110 lb 1:0ed3c684:::obj270:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0ed3c684:::obj270:head) v3 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed26813:::obj311:head local-lis/les=109/110 n=47 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0ed3c684:::obj270:head) 2026-02-11T23:31:17.133+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0ed3c684:::obj270:head) v3 finish latency 0.000015 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.133+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4f00 con 0x3752000 2026-02-11T23:31:17.133+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0ee9ae15:::obj4:head)]) -- 0x4c23200 con 0x3752000 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.133+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 134 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c3e9c0 con 0x3752000 2026-02-11T23:31:17.133+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.133+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.133+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 135 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0ee9ae15:::obj4:head) ==== 1060+0+0 (unknown 0 0 0) 0x4c33b80 con 0x3752000 2026-02-11T23:31:17.133+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0ee9ae15:::obj4:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.133+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0ee9ae15:::obj4:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0ee9ae15:::obj4:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0efd4012:::obj445:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0efd4012:::obj445:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0efd4012:::obj445:head 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0efd4012:::obj445:head v 107'445 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.133+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0ee9ae15:::obj4:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0ee9ae15:::obj4:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 110/110 lb 1:0ee9ae15:::obj4:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0ee9ae15:::obj4:head) v3 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ed3c684:::obj270:head local-lis/les=109/110 n=48 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0ee9ae15:::obj4:head) 2026-02-11T23:31:17.133+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.133+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0ee9ae15:::obj4:head) v3 finish latency 0.000014 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.133+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.133+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.133+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4300 con 0x3752000 2026-02-11T23:31:17.133+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0efd4012:::obj445:head)]) -- 0x4cb4000 con 0x3752000 2026-02-11T23:31:17.133+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.134+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 136 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c3eea0 con 0x3752000 2026-02-11T23:31:17.134+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.134+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.134+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.134+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 137 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0efd4012:::obj445:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c34100 con 0x3752000 2026-02-11T23:31:17.134+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0efd4012:::obj445:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.134+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.134+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0efd4012:::obj445:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0efd4012:::obj445:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.134+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.134+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0f33f807:::obj273:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.134+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0f33f807:::obj273:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0f33f807:::obj273:head 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0f33f807:::obj273:head v 107'273 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.134+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.134+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0efd4012:::obj445:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.134+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0efd4012:::obj445:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 110/110 lb 1:0efd4012:::obj445:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.134+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.134+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0efd4012:::obj445:head) v3 2026-02-11T23:31:17.134+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0ee9ae15:::obj4:head local-lis/les=109/110 n=49 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0efd4012:::obj445:head) 2026-02-11T23:31:17.134+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.134+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0efd4012:::obj445:head) v3 finish latency 0.000012 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.134+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.134+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.134+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af3800 con 0x3752000 2026-02-11T23:31:17.134+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0f33f807:::obj273:head)]) -- 0x4c23380 con 0x3752000 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.134+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 138 ==== MOSDPGPush(1.0 110/109 [PushOp(1:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c3f380 con 0x3752000 2026-02-11T23:31:17.134+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.134+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.134+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.134+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 139 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0f33f807:::obj273:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c34680 con 0x3752000 2026-02-11T23:31:17.134+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0f33f807:::obj273:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.134+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.134+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0f33f807:::obj273:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.134+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.134+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.134+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0f33f807:::obj273:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.134+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:0f45d4cb:::obj171:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.135+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:0f45d4cb:::obj171:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:0f45d4cb:::obj171:head 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:0f45d4cb:::obj171:head v 107'171 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.135+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.135+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0f33f807:::obj273:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0f33f807:::obj273:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 110/110 lb 1:0f33f807:::obj273:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0f33f807:::obj273:head) v3 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0efd4012:::obj445:head local-lis/les=109/110 n=50 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0f33f807:::obj273:head) 2026-02-11T23:31:17.135+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0f33f807:::obj273:head) v3 finish latency 0.000010 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.135+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.135+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2480 con 0x3752000 2026-02-11T23:31:17.135+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:0f45d4cb:::obj171:head)]) -- 0x4c23500 con 0x3752000 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.135+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 140 ==== MOSDPGPush(1.0 110/109 [PushOp(1:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x4c3f860 con 0x3752000 2026-02-11T23:31:17.135+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.135+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.135+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.135+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 141 ==== pg_backfill(progress 1.0 e 110/110 lb 1:0f45d4cb:::obj171:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c34c00 con 0x3752000 2026-02-11T23:31:17.135+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:0f45d4cb:::obj171:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.135+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.135+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0f45d4cb:::obj171:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.135+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.135+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0f45d4cb:::obj171:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:104778fc:::obj2:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.135+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:104778fc:::obj2:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:104778fc:::obj2:head 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:104778fc:::obj2:head v 107'2 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.135+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.135+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.135+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:0f45d4cb:::obj171:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0f45d4cb:::obj171:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 110/110 lb 1:0f45d4cb:::obj171:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:0f45d4cb:::obj171:head) v3 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f33f807:::obj273:head local-lis/les=109/110 n=51 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:0f45d4cb:::obj171:head) 2026-02-11T23:31:17.135+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:0f45d4cb:::obj171:head) v3 finish latency 0.000013 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.135+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.135+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.135+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2300 con 0x3752000 2026-02-11T23:31:17.135+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:104778fc:::obj2:head)]) -- 0x4c23680 con 0x3752000 2026-02-11T23:31:17.135+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.136+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 142 ==== MOSDPGPush(1.0 110/109 [PushOp(1:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4c3fd40 con 0x3752000 2026-02-11T23:31:17.136+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.136+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.136+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.136+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 143 ==== pg_backfill(progress 1.0 e 110/110 lb 1:104778fc:::obj2:head) ==== 1060+0+0 (unknown 0 0 0) 0x4c35180 con 0x3752000 2026-02-11T23:31:17.136+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:104778fc:::obj2:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.136+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.136+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:104778fc:::obj2:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.136+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.136+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.136+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.136+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:104778fc:::obj2:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.136+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.136+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.136+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.136+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.136+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:105983a6:::obj88:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.136+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:105983a6:::obj88:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.136+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:105983a6:::obj88:head 2026-02-11T23:31:17.136+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:105983a6:::obj88:head v 107'88 2026-02-11T23:31:17.136+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.136+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.136+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.136+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.136+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.136+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:104778fc:::obj2:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.136+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:104778fc:::obj2:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 110/110 lb 1:104778fc:::obj2:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.136+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.136+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:104778fc:::obj2:head) v3 2026-02-11T23:31:17.136+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:0f45d4cb:::obj171:head local-lis/les=109/110 n=52 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:104778fc:::obj2:head) 2026-02-11T23:31:17.136+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.136+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:104778fc:::obj2:head) v3 finish latency 0.000011 2026-02-11T23:31:17.136+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.136+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.136+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.136+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.136+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.136+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.136+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.136+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2900 con 0x3752000 2026-02-11T23:31:17.136+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.136+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:105983a6:::obj88:head)]) -- 0x4c22600 con 0x3752000 2026-02-11T23:31:17.136+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.136+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 144 ==== MOSDPGPush(1.0 110/109 [PushOp(1:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bd8340 con 0x3752000 2026-02-11T23:31:17.136+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.136+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.137+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 145 ==== pg_backfill(progress 1.0 e 110/110 lb 1:105983a6:::obj88:head) ==== 1061+0+0 (unknown 0 0 0) 0x4c35700 con 0x3752000 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.137+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:105983a6:::obj88:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.137+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:105983a6:::obj88:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:105983a6:::obj88:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:10754df1:::obj159:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:10754df1:::obj159:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:10754df1:::obj159:head 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:10754df1:::obj159:head v 107'159 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.137+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:105983a6:::obj88:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:105983a6:::obj88:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:105983a6:::obj88:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:105983a6:::obj88:head) v3 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:104778fc:::obj2:head local-lis/les=109/110 n=53 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:105983a6:::obj88:head) 2026-02-11T23:31:17.137+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:105983a6:::obj88:head) v3 finish latency 0.000013 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.137+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af3680 con 0x3752000 2026-02-11T23:31:17.137+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:10754df1:::obj159:head)]) -- 0x4c23980 con 0x3752000 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.137+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 146 ==== MOSDPGPush(1.0 110/109 [PushOp(1:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bd8820 con 0x3752000 2026-02-11T23:31:17.137+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.137+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.137+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 147 ==== pg_backfill(progress 1.0 e 110/110 lb 1:10754df1:::obj159:head) ==== 1062+0+0 (unknown 0 0 0) 0x4cb8000 con 0x3752000 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.137+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:10754df1:::obj159:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.137+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10754df1:::obj159:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10754df1:::obj159:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:10b422cc:::obj197:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:10b422cc:::obj197:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:10b422cc:::obj197:head 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:10b422cc:::obj197:head v 107'197 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.137+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10754df1:::obj159:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:10754df1:::obj159:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 110/110 lb 1:10754df1:::obj159:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:10754df1:::obj159:head) v3 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:105983a6:::obj88:head local-lis/les=109/110 n=54 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:10754df1:::obj159:head) 2026-02-11T23:31:17.137+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.137+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:10754df1:::obj159:head) v3 finish latency 0.000014 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.137+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.137+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.137+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2300 con 0x3752000 2026-02-11T23:31:17.137+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:10b422cc:::obj197:head)]) -- 0x4ba2000 con 0x3752000 2026-02-11T23:31:17.137+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.138+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 148 ==== MOSDPGPush(1.0 110/109 [PushOp(1:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bd8d00 con 0x3752000 2026-02-11T23:31:17.138+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.138+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.138+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 149 ==== pg_backfill(progress 1.0 e 110/110 lb 1:10b422cc:::obj197:head) ==== 1062+0+0 (unknown 0 0 0) 0x4cb8580 con 0x3752000 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.138+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:10b422cc:::obj197:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.138+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10b422cc:::obj197:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000022 MOSDPGPush(1.0 110/109 [PushOp(1:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.138+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.138+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10b422cc:::obj197:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:10c2d3a1:::obj448:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:10c2d3a1:::obj448:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:10c2d3a1:::obj448:head 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:10c2d3a1:::obj448:head v 107'448 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.138+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.138+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.138+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10b422cc:::obj197:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.138+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:10b422cc:::obj197:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 110/110 lb 1:10b422cc:::obj197:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.138+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.138+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:10b422cc:::obj197:head) v3 2026-02-11T23:31:17.138+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10754df1:::obj159:head local-lis/les=109/110 n=55 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:10b422cc:::obj197:head) 2026-02-11T23:31:17.138+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.138+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:10b422cc:::obj197:head) v3 finish latency 0.000014 2026-02-11T23:31:17.138+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.138+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.138+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2780 con 0x3752000 2026-02-11T23:31:17.138+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:10c2d3a1:::obj448:head)]) -- 0x4ba2480 con 0x3752000 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.138+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 150 ==== MOSDPGPush(1.0 110/109 [PushOp(1:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bd91e0 con 0x3752000 2026-02-11T23:31:17.138+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.138+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.138+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 151 ==== pg_backfill(progress 1.0 e 110/110 lb 1:10c2d3a1:::obj448:head) ==== 1062+0+0 (unknown 0 0 0) 0x4cb8b00 con 0x3752000 2026-02-11T23:31:17.138+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:10c2d3a1:::obj448:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.138+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10c2d3a1:::obj448:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.138+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.138+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10c2d3a1:::obj448:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:10f4e680:::obj204:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.138+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:10f4e680:::obj204:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:10f4e680:::obj204:head 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:10f4e680:::obj204:head v 107'204 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.138+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.138+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.139+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.139+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10c2d3a1:::obj448:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:10c2d3a1:::obj448:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:10c2d3a1:::obj448:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:10c2d3a1:::obj448:head) v3 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10b422cc:::obj197:head local-lis/les=109/110 n=56 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:10c2d3a1:::obj448:head) 2026-02-11T23:31:17.139+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:10c2d3a1:::obj448:head) v3 finish latency 0.000014 2026-02-11T23:31:17.139+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.139+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.139+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.139+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2c00 con 0x3752000 2026-02-11T23:31:17.139+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:10f4e680:::obj204:head)]) -- 0x4ba2900 con 0x3752000 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.139+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 152 ==== MOSDPGPush(1.0 110/109 [PushOp(1:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bd96c0 con 0x3752000 2026-02-11T23:31:17.139+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.139+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.139+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.139+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 153 ==== pg_backfill(progress 1.0 e 110/110 lb 1:10f4e680:::obj204:head) ==== 1062+0+0 (unknown 0 0 0) 0x4cb9080 con 0x3752000 2026-02-11T23:31:17.139+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.139+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:10f4e680:::obj204:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.139+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10f4e680:::obj204:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.139+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.139+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10f4e680:::obj204:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:11426d25:::obj314:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.139+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:11426d25:::obj314:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:11426d25:::obj314:head 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:11426d25:::obj314:head v 107'314 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.139+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.139+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.139+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:10f4e680:::obj204:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.139+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:10f4e680:::obj204:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:10f4e680:::obj204:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.139+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.139+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:10f4e680:::obj204:head) v3 2026-02-11T23:31:17.139+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10c2d3a1:::obj448:head local-lis/les=109/110 n=57 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:10f4e680:::obj204:head) 2026-02-11T23:31:17.139+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.139+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:10f4e680:::obj204:head) v3 finish latency 0.000014 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.139+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.139+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.139+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2d80 con 0x3752000 2026-02-11T23:31:17.139+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:11426d25:::obj314:head)]) -- 0x4c23e00 con 0x3752000 2026-02-11T23:31:17.139+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.140+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 154 ==== MOSDPGPush(1.0 110/109 [PushOp(1:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4bd9ba0 con 0x3752000 2026-02-11T23:31:17.140+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.140+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.140+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 155 ==== pg_backfill(progress 1.0 e 110/110 lb 1:11426d25:::obj314:head) ==== 1062+0+0 (unknown 0 0 0) 0x4cb9600 con 0x3752000 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.140+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:11426d25:::obj314:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.140+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:11426d25:::obj314:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:11426d25:::obj314:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:11a85aa0:::obj43:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:11a85aa0:::obj43:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:11a85aa0:::obj43:head 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:11a85aa0:::obj43:head v 107'43 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.140+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:11426d25:::obj314:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:11426d25:::obj314:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 110/110 lb 1:11426d25:::obj314:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:11426d25:::obj314:head) v3 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:10f4e680:::obj204:head local-lis/les=109/110 n=58 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:11426d25:::obj314:head) 2026-02-11T23:31:17.140+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:11426d25:::obj314:head) v3 finish latency 0.000014 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.140+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3080 con 0x3752000 2026-02-11T23:31:17.140+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:11a85aa0:::obj43:head)]) -- 0x4ba2f00 con 0x3752000 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.140+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 156 ==== MOSDPGPush(1.0 110/109 [PushOp(1:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bf01a0 con 0x3752000 2026-02-11T23:31:17.140+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 110 2026-02-11T23:31:17.140+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.140+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 157 ==== pg_backfill(progress 1.0 e 110/110 lb 1:11a85aa0:::obj43:head) ==== 1061+0+0 (unknown 0 0 0) 0x4cb9b80 con 0x3752000 2026-02-11T23:31:17.140+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:11a85aa0:::obj43:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.140+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:11a85aa0:::obj43:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:11a85aa0:::obj43:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:11bee109:::obj319:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:11bee109:::obj319:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:11bee109:::obj319:head 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:11bee109:::obj319:head v 107'319 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.140+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.140+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.140+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.140+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:11a85aa0:::obj43:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:11a85aa0:::obj43:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 110/110 lb 1:11a85aa0:::obj43:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:11a85aa0:::obj43:head) v3 2026-02-11T23:31:17.140+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11426d25:::obj314:head local-lis/les=109/110 n=59 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:11a85aa0:::obj43:head) 2026-02-11T23:31:17.140+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.141+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:11a85aa0:::obj43:head) v3 finish latency 0.000016 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.141+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.141+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.141+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3200 con 0x3752000 2026-02-11T23:31:17.141+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:11bee109:::obj319:head)]) -- 0x4cb4900 con 0x3752000 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.141+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 158 ==== MOSDPGPush(1.0 110/109 [PushOp(1:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bf0680 con 0x3752000 2026-02-11T23:31:17.141+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.141+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.141+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.141+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 159 ==== pg_backfill(progress 1.0 e 110/110 lb 1:11bee109:::obj319:head) ==== 1062+0+0 (unknown 0 0 0) 0x4cba100 con 0x3752000 2026-02-11T23:31:17.141+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.141+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:11bee109:::obj319:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.141+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:11bee109:::obj319:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.141+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.141+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.141+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.141+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.141+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.141+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:11bee109:::obj319:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.141+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:121cd428:::obj443:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.141+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:121cd428:::obj443:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.141+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:121cd428:::obj443:head 2026-02-11T23:31:17.141+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:121cd428:::obj443:head v 107'443 2026-02-11T23:31:17.141+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.141+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.141+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.141+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:11bee109:::obj319:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.141+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:11bee109:::obj319:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 110/110 lb 1:11bee109:::obj319:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.141+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.141+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:11bee109:::obj319:head) v3 2026-02-11T23:31:17.141+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11a85aa0:::obj43:head local-lis/les=109/110 n=60 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:11bee109:::obj319:head) 2026-02-11T23:31:17.141+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.141+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:11bee109:::obj319:head) v3 finish latency 0.000013 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.141+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.141+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.141+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2300 con 0x3752000 2026-02-11T23:31:17.141+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:121cd428:::obj443:head)]) -- 0x4cb5200 con 0x3752000 2026-02-11T23:31:17.141+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.142+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 160 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4bf0b60 con 0x3752000 2026-02-11T23:31:17.142+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.142+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.142+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 161 ==== pg_backfill(progress 1.0 e 110/110 lb 1:121cd428:::obj443:head) ==== 1062+0+0 (unknown 0 0 0) 0x4cba680 con 0x3752000 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.142+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:121cd428:::obj443:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.142+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:121cd428:::obj443:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(1.0 110/109 [PushOp(1:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:121cd428:::obj443:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1243fe1a:::obj21:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1243fe1a:::obj21:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1243fe1a:::obj21:head 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1243fe1a:::obj21:head v 107'21 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.142+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:121cd428:::obj443:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:121cd428:::obj443:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 110/110 lb 1:121cd428:::obj443:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:121cd428:::obj443:head) v3 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:11bee109:::obj319:head local-lis/les=109/110 n=61 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:121cd428:::obj443:head) 2026-02-11T23:31:17.142+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:121cd428:::obj443:head) v3 finish latency 0.000015 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.142+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2780 con 0x3752000 2026-02-11T23:31:17.142+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1243fe1a:::obj21:head)]) -- 0x4ba2000 con 0x3752000 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.142+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 162 ==== MOSDPGPush(1.0 110/109 [PushOp(1:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bf1040 con 0x3752000 2026-02-11T23:31:17.142+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.142+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.142+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 163 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1243fe1a:::obj21:head) ==== 1061+0+0 (unknown 0 0 0) 0x4cbac00 con 0x3752000 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.142+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1243fe1a:::obj21:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.142+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1243fe1a:::obj21:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1243fe1a:::obj21:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:124a99b2:::obj215:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:124a99b2:::obj215:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:124a99b2:::obj215:head 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:124a99b2:::obj215:head v 107'215 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.142+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.142+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.142+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1243fe1a:::obj21:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1243fe1a:::obj21:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 110/110 lb 1:1243fe1a:::obj21:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1243fe1a:::obj21:head) v3 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:121cd428:::obj443:head local-lis/les=109/110 n=62 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1243fe1a:::obj21:head) 2026-02-11T23:31:17.142+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.142+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1243fe1a:::obj21:head) v3 finish latency 0.000014 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.142+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.143+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.143+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.143+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2480 con 0x3752000 2026-02-11T23:31:17.143+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:124a99b2:::obj215:head)]) -- 0x4c22900 con 0x3752000 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.143+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 164 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bf1520 con 0x3752000 2026-02-11T23:31:17.143+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.143+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.143+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 165 ==== pg_backfill(progress 1.0 e 110/110 lb 1:124a99b2:::obj215:head) ==== 1062+0+0 (unknown 0 0 0) 0x4cbb180 con 0x3752000 2026-02-11T23:31:17.143+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:124a99b2:::obj215:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.143+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:124a99b2:::obj215:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.143+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.143+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:124a99b2:::obj215:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.143+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.143+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.143+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.143+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.143+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1405054b:::obj129:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1405054b:::obj129:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.143+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1405054b:::obj129:head 2026-02-11T23:31:17.143+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1405054b:::obj129:head v 107'129 2026-02-11T23:31:17.143+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.143+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.143+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.143+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.143+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:124a99b2:::obj215:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.143+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:124a99b2:::obj215:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 110/110 lb 1:124a99b2:::obj215:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.143+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.143+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:124a99b2:::obj215:head) v3 2026-02-11T23:31:17.143+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1243fe1a:::obj21:head local-lis/les=109/110 n=63 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:124a99b2:::obj215:head) 2026-02-11T23:31:17.143+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.143+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:124a99b2:::obj215:head) v3 finish latency 0.000014 2026-02-11T23:31:17.143+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.143+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.143+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.143+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3500 con 0x3752000 2026-02-11T23:31:17.143+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1405054b:::obj129:head)]) -- 0x4ba2c00 con 0x3752000 2026-02-11T23:31:17.143+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.144+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 166 ==== MOSDPGPush(1.0 110/109 [PushOp(1:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4bf1a00 con 0x3752000 2026-02-11T23:31:17.144+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.144+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.144+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 167 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1405054b:::obj129:head) ==== 1062+0+0 (unknown 0 0 0) 0x4cbb700 con 0x3752000 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.144+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1405054b:::obj129:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 110 2026-02-11T23:31:17.144+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1405054b:::obj129:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 110/109 [PushOp(1:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1405054b:::obj129:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:147354e0:::obj436:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:147354e0:::obj436:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:147354e0:::obj436:head 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:147354e0:::obj436:head v 107'436 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.144+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1405054b:::obj129:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1405054b:::obj129:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 110/110 lb 1:1405054b:::obj129:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1405054b:::obj129:head) v3 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:124a99b2:::obj215:head local-lis/les=109/110 n=64 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1405054b:::obj129:head) 2026-02-11T23:31:17.144+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1405054b:::obj129:head) v3 finish latency 0.000013 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.144+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3680 con 0x3752000 2026-02-11T23:31:17.144+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:147354e0:::obj436:head)]) -- 0x4c22f00 con 0x3752000 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.144+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 168 ==== MOSDPGPush(1.0 110/109 [PushOp(1:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c04000 con 0x3752000 2026-02-11T23:31:17.144+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.144+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.144+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 169 ==== pg_backfill(progress 1.0 e 110/110 lb 1:147354e0:::obj436:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c06000 con 0x3752000 2026-02-11T23:31:17.144+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:147354e0:::obj436:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.144+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:147354e0:::obj436:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:147354e0:::obj436:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:147b5010:::obj178:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:147b5010:::obj178:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:147b5010:::obj178:head 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:147b5010:::obj178:head v 107'178 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.144+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.144+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:147354e0:::obj436:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:147354e0:::obj436:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:147354e0:::obj436:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:147354e0:::obj436:head) v3 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1405054b:::obj129:head local-lis/les=109/110 n=65 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:147354e0:::obj436:head) 2026-02-11T23:31:17.144+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:147354e0:::obj436:head) v3 finish latency 0.000010 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.144+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.144+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.144+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23200 con 0x3752000 2026-02-11T23:31:17.144+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:147b5010:::obj178:head)]) -- 0x4af2480 con 0x3752000 2026-02-11T23:31:17.144+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.145+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 170 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c044e0 con 0x3752000 2026-02-11T23:31:17.145+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.145+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.145+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.145+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 171 ==== pg_backfill(progress 1.0 e 110/110 lb 1:147b5010:::obj178:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c06580 con 0x3752000 2026-02-11T23:31:17.145+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:147b5010:::obj178:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.145+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.145+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:147b5010:::obj178:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.145+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.145+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.145+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 110/109 [PushOp(1:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.145+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.145+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.145+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.145+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.145+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:147b5010:::obj178:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.145+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1503467e:::obj118:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.145+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1503467e:::obj118:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.145+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1503467e:::obj118:head 2026-02-11T23:31:17.145+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1503467e:::obj118:head v 107'118 2026-02-11T23:31:17.145+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.145+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.145+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.145+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.145+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.145+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.145+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:147b5010:::obj178:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.145+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:147b5010:::obj178:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 110/110 lb 1:147b5010:::obj178:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.145+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.145+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:147b5010:::obj178:head) v3 2026-02-11T23:31:17.145+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147354e0:::obj436:head local-lis/les=109/110 n=66 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:147b5010:::obj178:head) 2026-02-11T23:31:17.145+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.145+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:147b5010:::obj178:head) v3 finish latency 0.000015 2026-02-11T23:31:17.145+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.145+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.145+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.145+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.145+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.145+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.145+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23680 con 0x3752000 2026-02-11T23:31:17.145+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1503467e:::obj118:head)]) -- 0x4c23380 con 0x3752000 2026-02-11T23:31:17.145+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.146+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 172 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c049c0 con 0x3752000 2026-02-11T23:31:17.146+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.146+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.146+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 173 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1503467e:::obj118:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c06b00 con 0x3752000 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.146+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1503467e:::obj118:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.146+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1503467e:::obj118:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 110/109 [PushOp(1:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1503467e:::obj118:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1547e7aa:::obj466:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1547e7aa:::obj466:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1547e7aa:::obj466:head 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1547e7aa:::obj466:head v 107'466 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.146+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1503467e:::obj118:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1503467e:::obj118:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 110/110 lb 1:1503467e:::obj118:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1503467e:::obj118:head) v3 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:147b5010:::obj178:head local-lis/les=109/110 n=67 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1503467e:::obj118:head) 2026-02-11T23:31:17.146+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1503467e:::obj118:head) v3 finish latency 0.000014 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.146+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c22600 con 0x3752000 2026-02-11T23:31:17.146+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1547e7aa:::obj466:head)]) -- 0x4af2900 con 0x3752000 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.146+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 174 ==== MOSDPGPush(1.0 110/109 [PushOp(1:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c04ea0 con 0x3752000 2026-02-11T23:31:17.146+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 110 2026-02-11T23:31:17.146+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.146+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 175 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1547e7aa:::obj466:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c07080 con 0x3752000 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.146+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1547e7aa:::obj466:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.146+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1547e7aa:::obj466:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1547e7aa:::obj466:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:170a838d:::obj199:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:170a838d:::obj199:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:170a838d:::obj199:head 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:170a838d:::obj199:head v 107'199 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.146+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.146+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1547e7aa:::obj466:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1547e7aa:::obj466:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 1.0 e 110/110 lb 1:1547e7aa:::obj466:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1547e7aa:::obj466:head) v3 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1503467e:::obj118:head local-lis/les=109/110 n=68 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1547e7aa:::obj466:head) 2026-02-11T23:31:17.146+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1547e7aa:::obj466:head) v3 finish latency 0.000014 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.146+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.146+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.146+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23980 con 0x3752000 2026-02-11T23:31:17.146+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:170a838d:::obj199:head)]) -- 0x4cb4d80 con 0x3752000 2026-02-11T23:31:17.146+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.147+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 176 ==== MOSDPGPush(1.0 110/109 [PushOp(1:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c05380 con 0x3752000 2026-02-11T23:31:17.147+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.147+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.147+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.147+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 177 ==== pg_backfill(progress 1.0 e 110/110 lb 1:170a838d:::obj199:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c07600 con 0x3752000 2026-02-11T23:31:17.147+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.147+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:170a838d:::obj199:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.147+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:170a838d:::obj199:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.147+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.147+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.147+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 110/109 [PushOp(1:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.147+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:170a838d:::obj199:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.147+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.147+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.147+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:171b715e:::obj570:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.147+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:171b715e:::obj570:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.147+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:171b715e:::obj570:head 2026-02-11T23:31:17.147+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:171b715e:::obj570:head v 107'570 2026-02-11T23:31:17.147+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.147+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.147+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.147+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:170a838d:::obj199:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.147+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:170a838d:::obj199:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 110/110 lb 1:170a838d:::obj199:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.147+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.147+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:170a838d:::obj199:head) v3 2026-02-11T23:31:17.147+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1547e7aa:::obj466:head local-lis/les=109/110 n=69 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:170a838d:::obj199:head) 2026-02-11T23:31:17.147+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.147+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:170a838d:::obj199:head) v3 finish latency 0.000017 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.147+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.147+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.147+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3800 con 0x3752000 2026-02-11T23:31:17.147+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:171b715e:::obj570:head)]) -- 0x4cb5080 con 0x3752000 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.147+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 178 ==== MOSDPGPush(1.0 110/109 [PushOp(1:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c05860 con 0x3752000 2026-02-11T23:31:17.147+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.147+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.147+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 179 ==== pg_backfill(progress 1.0 e 110/110 lb 1:171b715e:::obj570:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c07b80 con 0x3752000 2026-02-11T23:31:17.147+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:171b715e:::obj570:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.147+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:171b715e:::obj570:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.147+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.147+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.147+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.147+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.147+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.147+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:171b715e:::obj570:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.147+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.147+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:17371e94:::obj362:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:17371e94:::obj362:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:17371e94:::obj362:head 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:17371e94:::obj362:head v 107'362 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.148+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.148+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.148+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:171b715e:::obj570:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.148+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:171b715e:::obj570:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 110/110 lb 1:171b715e:::obj570:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.148+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.148+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:171b715e:::obj570:head) v3 2026-02-11T23:31:17.148+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:170a838d:::obj199:head local-lis/les=109/110 n=70 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:171b715e:::obj570:head) 2026-02-11T23:31:17.148+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.148+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:171b715e:::obj570:head) v3 finish latency 0.000014 2026-02-11T23:31:17.148+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.148+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.148+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3c80 con 0x3752000 2026-02-11T23:31:17.148+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:17371e94:::obj362:head)]) -- 0x4ba3980 con 0x3752000 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.148+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 180 ==== MOSDPGPush(1.0 110/109 [PushOp(1:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4c05d40 con 0x3752000 2026-02-11T23:31:17.148+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.148+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.148+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 181 ==== pg_backfill(progress 1.0 e 110/110 lb 1:17371e94:::obj362:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c08100 con 0x3752000 2026-02-11T23:31:17.148+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:17371e94:::obj362:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.148+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:17371e94:::obj362:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 110/109 [PushOp(1:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.148+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.148+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:17371e94:::obj362:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:18100969:::obj255:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:18100969:::obj255:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:18100969:::obj255:head 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:18100969:::obj255:head v 107'255 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.148+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.148+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.148+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:17371e94:::obj362:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.148+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:17371e94:::obj362:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 110/110 lb 1:17371e94:::obj362:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.148+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.148+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:17371e94:::obj362:head) v3 2026-02-11T23:31:17.148+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:171b715e:::obj570:head local-lis/les=109/110 n=71 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:17371e94:::obj362:head) 2026-02-11T23:31:17.148+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.148+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:17371e94:::obj362:head) v3 finish latency 0.000014 2026-02-11T23:31:17.148+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.148+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.148+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.148+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2d80 con 0x3752000 2026-02-11T23:31:17.148+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:18100969:::obj255:head)]) -- 0x4ba3e00 con 0x3752000 2026-02-11T23:31:17.148+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.149+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 182 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d36340 con 0x3752000 2026-02-11T23:31:17.149+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.149+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.149+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.149+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 183 ==== pg_backfill(progress 1.0 e 110/110 lb 1:18100969:::obj255:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c08680 con 0x3752000 2026-02-11T23:31:17.149+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.149+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.149+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:18100969:::obj255:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.149+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.149+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:18100969:::obj255:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.149+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.149+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 110/109 [PushOp(1:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.149+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:18100969:::obj255:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.149+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.149+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.149+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.149+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1842fa44:::obj284:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.149+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1842fa44:::obj284:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.149+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1842fa44:::obj284:head 2026-02-11T23:31:17.149+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1842fa44:::obj284:head v 107'284 2026-02-11T23:31:17.149+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.149+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.149+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.149+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.149+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.149+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:18100969:::obj255:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.149+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:18100969:::obj255:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 110/110 lb 1:18100969:::obj255:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.149+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.149+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:18100969:::obj255:head) v3 2026-02-11T23:31:17.149+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:17371e94:::obj362:head local-lis/les=109/110 n=72 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:18100969:::obj255:head) 2026-02-11T23:31:17.149+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.149+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:18100969:::obj255:head) v3 finish latency 0.000014 2026-02-11T23:31:17.149+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.149+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.149+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.149+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.149+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.149+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.149+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.149+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3080 con 0x3752000 2026-02-11T23:31:17.149+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1842fa44:::obj284:head)]) -- 0x4cb4300 con 0x3752000 2026-02-11T23:31:17.149+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.149+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 184 ==== MOSDPGPush(1.0 110/109 [PushOp(1:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d36820 con 0x3752000 2026-02-11T23:31:17.150+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.150+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.150+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 185 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1842fa44:::obj284:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c08c00 con 0x3752000 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.150+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1842fa44:::obj284:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.150+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1842fa44:::obj284:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 110/109 [PushOp(1:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1842fa44:::obj284:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:18b572c6:::obj440:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:18b572c6:::obj440:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:18b572c6:::obj440:head 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:18b572c6:::obj440:head v 107'440 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.150+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1842fa44:::obj284:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1842fa44:::obj284:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 110/110 lb 1:1842fa44:::obj284:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1842fa44:::obj284:head) v3 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18100969:::obj255:head local-lis/les=109/110 n=73 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1842fa44:::obj284:head) 2026-02-11T23:31:17.150+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1842fa44:::obj284:head) v3 finish latency 0.000014 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.150+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2f00 con 0x3752000 2026-02-11T23:31:17.150+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:18b572c6:::obj440:head)]) -- 0x4cb4900 con 0x3752000 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.150+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 186 ==== MOSDPGPush(1.0 110/109 [PushOp(1:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4d36d00 con 0x3752000 2026-02-11T23:31:17.150+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.150+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.150+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 187 ==== pg_backfill(progress 1.0 e 110/110 lb 1:18b572c6:::obj440:head) ==== 1062+0+0 (unknown 0 0 0) 0x4c09180 con 0x3752000 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.150+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:18b572c6:::obj440:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.150+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:18b572c6:::obj440:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:18b572c6:::obj440:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:18c50d58:::obj63:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:18c50d58:::obj63:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:18c50d58:::obj63:head 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:18c50d58:::obj63:head v 107'63 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.150+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.150+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:18b572c6:::obj440:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:18b572c6:::obj440:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:18b572c6:::obj440:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:18b572c6:::obj440:head) v3 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1842fa44:::obj284:head local-lis/les=109/110 n=74 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:18b572c6:::obj440:head) 2026-02-11T23:31:17.150+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:18b572c6:::obj440:head) v3 finish latency 0.000016 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.150+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.150+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.150+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3200 con 0x3752000 2026-02-11T23:31:17.150+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:18c50d58:::obj63:head)]) -- 0x4cb4d80 con 0x3752000 2026-02-11T23:31:17.150+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.151+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 188 ==== MOSDPGPush(1.0 110/109 [PushOp(1:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d371e0 con 0x3752000 2026-02-11T23:31:17.151+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.151+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.151+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.151+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 189 ==== pg_backfill(progress 1.0 e 110/110 lb 1:18c50d58:::obj63:head) ==== 1061+0+0 (unknown 0 0 0) 0x4c09700 con 0x3752000 2026-02-11T23:31:17.151+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:18c50d58:::obj63:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.151+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.151+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:18c50d58:::obj63:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.151+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.151+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.151+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.151+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:18c50d58:::obj63:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.151+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.151+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.151+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.151+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.151+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:19299046:::obj541:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.151+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:19299046:::obj541:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.151+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:19299046:::obj541:head 2026-02-11T23:31:17.151+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:19299046:::obj541:head v 107'541 2026-02-11T23:31:17.151+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.151+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.151+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:17.151+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.151+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.151+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:18c50d58:::obj63:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.151+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:18c50d58:::obj63:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 110/110 lb 1:18c50d58:::obj63:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.151+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.151+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:18c50d58:::obj63:head) v3 2026-02-11T23:31:17.151+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18b572c6:::obj440:head local-lis/les=109/110 n=75 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:18c50d58:::obj63:head) 2026-02-11T23:31:17.151+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.151+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:18c50d58:::obj63:head) v3 finish latency 0.000017 2026-02-11T23:31:17.151+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.151+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.151+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.151+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.151+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.151+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.151+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.151+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2300 con 0x3752000 2026-02-11T23:31:17.151+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:19299046:::obj541:head)]) -- 0x4cb5080 con 0x3752000 2026-02-11T23:31:17.151+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.152+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 190 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4d376c0 con 0x3752000 2026-02-11T23:31:17.152+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000021 epoch 110 2026-02-11T23:31:17.152+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.152+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 191 ==== pg_backfill(progress 1.0 e 110/110 lb 1:19299046:::obj541:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d4c000 con 0x3752000 2026-02-11T23:31:17.152+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:19299046:::obj541:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 110 2026-02-11T23:31:17.152+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:19299046:::obj541:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 110/109 [PushOp(1:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:19299046:::obj541:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1a1150ce:::obj60:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1a1150ce:::obj60:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1a1150ce:::obj60:head 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1a1150ce:::obj60:head v 107'60 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.152+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:19299046:::obj541:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:19299046:::obj541:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 110/110 lb 1:19299046:::obj541:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:19299046:::obj541:head) v3 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:18c50d58:::obj63:head local-lis/les=109/110 n=76 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:19299046:::obj541:head) 2026-02-11T23:31:17.152+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:19299046:::obj541:head) v3 finish latency 0.000016 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.152+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23e00 con 0x3752000 2026-02-11T23:31:17.152+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1a1150ce:::obj60:head)]) -- 0x4ba2000 con 0x3752000 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.152+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 192 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d37ba0 con 0x3752000 2026-02-11T23:31:17.152+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 110 2026-02-11T23:31:17.152+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.152+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 193 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1a1150ce:::obj60:head) ==== 1061+0+0 (unknown 0 0 0) 0x4d4c580 con 0x3752000 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.152+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a1150ce:::obj60:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.152+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a1150ce:::obj60:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 110/109 [PushOp(1:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a1150ce:::obj60:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1a20990d:::obj103:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1a20990d:::obj103:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1a20990d:::obj103:head 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1a20990d:::obj103:head v 107'103 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.152+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a1150ce:::obj60:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a1150ce:::obj60:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 110/110 lb 1:1a1150ce:::obj60:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1a1150ce:::obj60:head) v3 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:19299046:::obj541:head local-lis/les=109/110 n=77 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1a1150ce:::obj60:head) 2026-02-11T23:31:17.152+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.152+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a1150ce:::obj60:head) v3 finish latency 0.000014 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.152+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.152+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23200 con 0x3752000 2026-02-11T23:31:17.152+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.152+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1a20990d:::obj103:head)]) -- 0x4c22900 con 0x3752000 2026-02-11T23:31:17.152+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.153+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 194 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d521a0 con 0x3752000 2026-02-11T23:31:17.153+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 110 2026-02-11T23:31:17.153+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.153+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.153+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 195 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1a20990d:::obj103:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d4cb00 con 0x3752000 2026-02-11T23:31:17.153+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a20990d:::obj103:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.153+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.153+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a20990d:::obj103:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.153+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.153+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.153+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.153+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.153+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.153+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.153+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a20990d:::obj103:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.153+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.153+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1a2eef70:::obj126:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.153+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1a2eef70:::obj126:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.153+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1a2eef70:::obj126:head 2026-02-11T23:31:17.153+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1a2eef70:::obj126:head v 107'126 2026-02-11T23:31:17.153+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.153+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.153+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.153+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.153+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.153+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a20990d:::obj103:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.153+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a20990d:::obj103:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 110/110 lb 1:1a20990d:::obj103:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.153+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.153+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1a20990d:::obj103:head) v3 2026-02-11T23:31:17.153+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a1150ce:::obj60:head local-lis/les=109/110 n=78 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1a20990d:::obj103:head) 2026-02-11T23:31:17.153+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.153+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a20990d:::obj103:head) v3 finish latency 0.000015 2026-02-11T23:31:17.153+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.153+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.153+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.153+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.153+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.153+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.153+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.153+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23680 con 0x3752000 2026-02-11T23:31:17.153+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1a2eef70:::obj126:head)]) -- 0x4ba2480 con 0x3752000 2026-02-11T23:31:17.153+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.153+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 196 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d52680 con 0x3752000 2026-02-11T23:31:17.153+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.153+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.153+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.153+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 197 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1a2eef70:::obj126:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d4d080 con 0x3752000 2026-02-11T23:31:17.153+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.153+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a2eef70:::obj126:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.153+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.154+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.154+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a2eef70:::obj126:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 110/109 [PushOp(1:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a2eef70:::obj126:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1a783fd9:::obj328:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.154+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1a783fd9:::obj328:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1a783fd9:::obj328:head 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1a783fd9:::obj328:head v 107'328 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.154+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.154+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a2eef70:::obj126:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a2eef70:::obj126:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 110/110 lb 1:1a2eef70:::obj126:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1a2eef70:::obj126:head) v3 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a20990d:::obj103:head local-lis/les=109/110 n=79 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1a2eef70:::obj126:head) 2026-02-11T23:31:17.154+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a2eef70:::obj126:head) v3 finish latency 0.000014 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.154+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.154+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23380 con 0x3752000 2026-02-11T23:31:17.154+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1a783fd9:::obj328:head)]) -- 0x4ba2c00 con 0x3752000 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.154+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 198 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4d52b60 con 0x3752000 2026-02-11T23:31:17.154+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.154+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.154+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.154+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 199 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1a783fd9:::obj328:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d4d600 con 0x3752000 2026-02-11T23:31:17.154+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a783fd9:::obj328:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.154+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.154+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a783fd9:::obj328:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.154+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.154+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a783fd9:::obj328:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1abd4411:::obj66:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.154+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1abd4411:::obj66:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1abd4411:::obj66:head 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1abd4411:::obj66:head v 107'66 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.154+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.154+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.154+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1a783fd9:::obj328:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a783fd9:::obj328:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 110/110 lb 1:1a783fd9:::obj328:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1a783fd9:::obj328:head) v3 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a2eef70:::obj126:head local-lis/les=109/110 n=80 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1a783fd9:::obj328:head) 2026-02-11T23:31:17.154+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1a783fd9:::obj328:head) v3 finish latency 0.000011 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.154+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.154+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.154+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c22600 con 0x3752000 2026-02-11T23:31:17.154+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1abd4411:::obj66:head)]) -- 0x4ba3800 con 0x3752000 2026-02-11T23:31:17.154+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.155+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 200 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4d53040 con 0x3752000 2026-02-11T23:31:17.155+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000019 epoch 110 2026-02-11T23:31:17.155+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.155+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.155+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.155+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 201 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1abd4411:::obj66:head) ==== 1061+0+0 (unknown 0 0 0) 0x4d4db80 con 0x3752000 2026-02-11T23:31:17.155+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.155+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.155+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.155+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1abd4411:::obj66:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 110 2026-02-11T23:31:17.155+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 110/109 [PushOp(1:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.155+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.155+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1abd4411:::obj66:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.155+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.155+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.155+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.155+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1abd4411:::obj66:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.155+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1b644371:::obj48:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.155+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1b644371:::obj48:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.155+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1b644371:::obj48:head 2026-02-11T23:31:17.155+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1b644371:::obj48:head v 107'48 2026-02-11T23:31:17.155+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.155+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.155+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.155+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.155+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.155+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1abd4411:::obj66:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.155+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1abd4411:::obj66:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 110/110 lb 1:1abd4411:::obj66:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.155+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.155+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1abd4411:::obj66:head) v3 2026-02-11T23:31:17.155+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1a783fd9:::obj328:head local-lis/les=109/110 n=81 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1abd4411:::obj66:head) 2026-02-11T23:31:17.155+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.155+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1abd4411:::obj66:head) v3 finish latency 0.000009 2026-02-11T23:31:17.155+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.155+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.155+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.155+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.155+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.155+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.155+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.155+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23980 con 0x3752000 2026-02-11T23:31:17.155+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1b644371:::obj48:head)]) -- 0x4d66000 con 0x3752000 2026-02-11T23:31:17.155+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.156+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 202 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4d53520 con 0x3752000 2026-02-11T23:31:17.156+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 110 2026-02-11T23:31:17.156+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.156+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 203 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1b644371:::obj48:head) ==== 1061+0+0 (unknown 0 0 0) 0x4d4e100 con 0x3752000 2026-02-11T23:31:17.156+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1b644371:::obj48:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.156+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1b644371:::obj48:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1b644371:::obj48:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1b6bffcc:::obj61:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1b6bffcc:::obj61:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1b6bffcc:::obj61:head 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1b6bffcc:::obj61:head v 107'61 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.156+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1b644371:::obj48:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1b644371:::obj48:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 110/110 lb 1:1b644371:::obj48:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1b644371:::obj48:head) v3 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1abd4411:::obj66:head local-lis/les=109/110 n=82 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1b644371:::obj48:head) 2026-02-11T23:31:17.156+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1b644371:::obj48:head) v3 finish latency 0.000009 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.156+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2480 con 0x3752000 2026-02-11T23:31:17.156+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1b6bffcc:::obj61:head)]) -- 0x4d66180 con 0x3752000 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.156+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 204 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d53a00 con 0x3752000 2026-02-11T23:31:17.156+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.156+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.156+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 205 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1b6bffcc:::obj61:head) ==== 1061+0+0 (unknown 0 0 0) 0x4d4e680 con 0x3752000 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.156+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1b6bffcc:::obj61:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 110 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.156+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1b6bffcc:::obj61:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1b6bffcc:::obj61:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1b6ee6bf:::obj501:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1b6ee6bf:::obj501:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1b6ee6bf:::obj501:head 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1b6ee6bf:::obj501:head v 107'501 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.156+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.156+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1b6bffcc:::obj61:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1b6bffcc:::obj61:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 110/110 lb 1:1b6bffcc:::obj61:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1b6bffcc:::obj61:head) v3 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b644371:::obj48:head local-lis/les=109/110 n=83 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1b6bffcc:::obj61:head) 2026-02-11T23:31:17.156+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1b6bffcc:::obj61:head) v3 finish latency 0.000008 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.156+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.156+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.156+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2900 con 0x3752000 2026-02-11T23:31:17.156+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1b6ee6bf:::obj501:head)]) -- 0x4d66300 con 0x3752000 2026-02-11T23:31:17.156+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.157+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 206 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d6e000 con 0x3752000 2026-02-11T23:31:17.157+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.157+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.157+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.157+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 207 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1b6ee6bf:::obj501:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d4ec00 con 0x3752000 2026-02-11T23:31:17.157+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.157+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1b6ee6bf:::obj501:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.157+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1b6ee6bf:::obj501:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.157+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.157+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.157+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 110/109 [PushOp(1:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.157+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.157+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.157+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1b6ee6bf:::obj501:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.157+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.157+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.157+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1bad8e21:::obj122:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.157+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1bad8e21:::obj122:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.157+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1bad8e21:::obj122:head 2026-02-11T23:31:17.157+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1bad8e21:::obj122:head v 107'122 2026-02-11T23:31:17.157+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.157+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.157+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.157+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.157+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.157+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1b6ee6bf:::obj501:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.157+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1b6ee6bf:::obj501:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 110/110 lb 1:1b6ee6bf:::obj501:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.157+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.157+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1b6ee6bf:::obj501:head) v3 2026-02-11T23:31:17.157+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6bffcc:::obj61:head local-lis/les=109/110 n=84 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1b6ee6bf:::obj501:head) 2026-02-11T23:31:17.157+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.157+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1b6ee6bf:::obj501:head) v3 finish latency 0.000009 2026-02-11T23:31:17.157+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.157+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.157+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.157+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.157+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.157+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.157+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.157+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23e00 con 0x3752000 2026-02-11T23:31:17.157+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1bad8e21:::obj122:head)]) -- 0x4d66600 con 0x3752000 2026-02-11T23:31:17.157+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.158+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 208 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d6e4e0 con 0x3752000 2026-02-11T23:31:17.158+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.158+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.158+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 209 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1bad8e21:::obj122:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d4f180 con 0x3752000 2026-02-11T23:31:17.158+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1bad8e21:::obj122:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.158+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1bad8e21:::obj122:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1bad8e21:::obj122:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1bb1e69e:::obj390:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1bb1e69e:::obj390:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1bb1e69e:::obj390:head 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1bb1e69e:::obj390:head v 107'390 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.158+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1bad8e21:::obj122:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1bad8e21:::obj122:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 110/110 lb 1:1bad8e21:::obj122:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1bad8e21:::obj122:head) v3 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1b6ee6bf:::obj501:head local-lis/les=109/110 n=85 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1bad8e21:::obj122:head) 2026-02-11T23:31:17.158+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1bad8e21:::obj122:head) v3 finish latency 0.000010 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.158+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3c80 con 0x3752000 2026-02-11T23:31:17.158+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1bb1e69e:::obj390:head)]) -- 0x4d66900 con 0x3752000 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.158+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 210 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d6e9c0 con 0x3752000 2026-02-11T23:31:17.158+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.158+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.158+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 211 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1bb1e69e:::obj390:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d4f700 con 0x3752000 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.158+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1bb1e69e:::obj390:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.158+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1bb1e69e:::obj390:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 110/109 [PushOp(1:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1bb1e69e:::obj390:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1c47bf8e:::obj558:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1c47bf8e:::obj558:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1c47bf8e:::obj558:head 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1c47bf8e:::obj558:head v 107'558 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.158+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.158+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1bb1e69e:::obj390:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1bb1e69e:::obj390:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 110/110 lb 1:1bb1e69e:::obj390:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1bb1e69e:::obj390:head) v3 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bad8e21:::obj122:head local-lis/les=109/110 n=86 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1bb1e69e:::obj390:head) 2026-02-11T23:31:17.158+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1bb1e69e:::obj390:head) v3 finish latency 0.000021 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.158+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.158+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.158+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3980 con 0x3752000 2026-02-11T23:31:17.158+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1c47bf8e:::obj558:head)]) -- 0x4d66c00 con 0x3752000 2026-02-11T23:31:17.158+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.159+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 212 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d6eea0 con 0x3752000 2026-02-11T23:31:17.159+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.159+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.159+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.159+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.159+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.159+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.159+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 213 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1c47bf8e:::obj558:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d76000 con 0x3752000 2026-02-11T23:31:17.159+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1c47bf8e:::obj558:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 110 2026-02-11T23:31:17.159+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1c47bf8e:::obj558:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.159+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.159+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000022 MOSDPGPush(1.0 110/109 [PushOp(1:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.159+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.159+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.159+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.159+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.159+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1c47bf8e:::obj558:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.159+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1cb33d95:::obj190:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.159+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1cb33d95:::obj190:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.159+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1cb33d95:::obj190:head 2026-02-11T23:31:17.159+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1cb33d95:::obj190:head v 107'190 2026-02-11T23:31:17.159+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.159+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.159+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.159+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.159+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.159+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1c47bf8e:::obj558:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.159+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1c47bf8e:::obj558:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:1c47bf8e:::obj558:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.159+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.159+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1c47bf8e:::obj558:head) v3 2026-02-11T23:31:17.159+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1bb1e69e:::obj390:head local-lis/les=109/110 n=87 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1c47bf8e:::obj558:head) 2026-02-11T23:31:17.159+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.159+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1c47bf8e:::obj558:head) v3 finish latency 0.000010 2026-02-11T23:31:17.159+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.159+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.159+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.159+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.159+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.159+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.159+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.159+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2d80 con 0x3752000 2026-02-11T23:31:17.159+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1cb33d95:::obj190:head)]) -- 0x4d66f00 con 0x3752000 2026-02-11T23:31:17.159+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.160+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 214 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d6f380 con 0x3752000 2026-02-11T23:31:17.160+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000017 epoch 110 2026-02-11T23:31:17.160+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.160+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 215 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1cb33d95:::obj190:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d76580 con 0x3752000 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.160+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cb33d95:::obj190:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 110 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.160+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cb33d95:::obj190:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 110/109 [PushOp(1:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cb33d95:::obj190:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1cd4c3d5:::obj142:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1cd4c3d5:::obj142:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1cd4c3d5:::obj142:head 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1cd4c3d5:::obj142:head v 107'142 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.160+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cb33d95:::obj190:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.160+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cb33d95:::obj190:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 110/110 lb 1:1cb33d95:::obj190:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.160+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.160+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1cb33d95:::obj190:head) v3 2026-02-11T23:31:17.160+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1c47bf8e:::obj558:head local-lis/les=109/110 n=88 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1cb33d95:::obj190:head) 2026-02-11T23:31:17.160+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.160+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cb33d95:::obj190:head) v3 finish latency 0.000010 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.160+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.160+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3e00 con 0x3752000 2026-02-11T23:31:17.160+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1cd4c3d5:::obj142:head)]) -- 0x4d67200 con 0x3752000 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.160+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 216 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d6f860 con 0x3752000 2026-02-11T23:31:17.160+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.160+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.160+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 217 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1cd4c3d5:::obj142:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d76b00 con 0x3752000 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.160+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cd4c3d5:::obj142:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.160+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cd4c3d5:::obj142:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000022 MOSDPGPush(1.0 110/109 [PushOp(1:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cd4c3d5:::obj142:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1cf1ad59:::obj156:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1cf1ad59:::obj156:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1cf1ad59:::obj156:head 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1cf1ad59:::obj156:head v 107'156 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.160+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.160+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cd4c3d5:::obj142:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.160+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cd4c3d5:::obj142:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 110/110 lb 1:1cd4c3d5:::obj142:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.160+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.160+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1cd4c3d5:::obj142:head) v3 2026-02-11T23:31:17.160+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cb33d95:::obj190:head local-lis/les=109/110 n=89 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1cd4c3d5:::obj142:head) 2026-02-11T23:31:17.160+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.160+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cd4c3d5:::obj142:head) v3 finish latency 0.000010 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.160+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.160+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.161+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.161+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3080 con 0x3752000 2026-02-11T23:31:17.161+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1cf1ad59:::obj156:head)]) -- 0x4d67500 con 0x3752000 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.161+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 218 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d6fd40 con 0x3752000 2026-02-11T23:31:17.161+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.161+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.161+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.161+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 219 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1cf1ad59:::obj156:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d77080 con 0x3752000 2026-02-11T23:31:17.161+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.161+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cf1ad59:::obj156:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.161+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cf1ad59:::obj156:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.161+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.161+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.161+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 110/109 [PushOp(1:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.161+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cf1ad59:::obj156:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.161+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.161+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.161+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1cff0951:::obj258:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.161+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1cff0951:::obj258:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.161+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1cff0951:::obj258:head 2026-02-11T23:31:17.161+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1cff0951:::obj258:head v 107'258 2026-02-11T23:31:17.161+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.161+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.161+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.161+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cf1ad59:::obj156:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.161+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cf1ad59:::obj156:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:1cf1ad59:::obj156:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.161+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.161+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1cf1ad59:::obj156:head) v3 2026-02-11T23:31:17.161+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cd4c3d5:::obj142:head local-lis/les=109/110 n=90 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1cf1ad59:::obj156:head) 2026-02-11T23:31:17.161+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.161+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cf1ad59:::obj156:head) v3 finish latency 0.000009 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.161+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.161+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.161+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2f00 con 0x3752000 2026-02-11T23:31:17.161+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1cff0951:::obj258:head)]) -- 0x4d67800 con 0x3752000 2026-02-11T23:31:17.161+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.162+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 220 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d44340 con 0x3752000 2026-02-11T23:31:17.162+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 110 2026-02-11T23:31:17.162+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.162+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 221 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1cff0951:::obj258:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d77600 con 0x3752000 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.162+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cff0951:::obj258:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.162+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cff0951:::obj258:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 110/109 [PushOp(1:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cff0951:::obj258:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1d2e941a:::obj186:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1d2e941a:::obj186:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1d2e941a:::obj186:head 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1d2e941a:::obj186:head v 107'186 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.162+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1cff0951:::obj258:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cff0951:::obj258:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:1cff0951:::obj258:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1cff0951:::obj258:head) v3 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cf1ad59:::obj156:head local-lis/les=109/110 n=91 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1cff0951:::obj258:head) 2026-02-11T23:31:17.162+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1cff0951:::obj258:head) v3 finish latency 0.000010 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.162+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3200 con 0x3752000 2026-02-11T23:31:17.162+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1d2e941a:::obj186:head)]) -- 0x4d66180 con 0x3752000 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.162+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 222 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4d44820 con 0x3752000 2026-02-11T23:31:17.162+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.162+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.162+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 223 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1d2e941a:::obj186:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d77b80 con 0x3752000 2026-02-11T23:31:17.162+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1d2e941a:::obj186:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.162+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1d2e941a:::obj186:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000022 MOSDPGPush(1.0 110/109 [PushOp(1:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1d2e941a:::obj186:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1dafe4f0:::obj47:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1dafe4f0:::obj47:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1dafe4f0:::obj47:head 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1dafe4f0:::obj47:head v 107'47 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.162+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.162+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1d2e941a:::obj186:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1d2e941a:::obj186:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 110/110 lb 1:1d2e941a:::obj186:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1d2e941a:::obj186:head) v3 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1cff0951:::obj258:head local-lis/les=109/110 n=92 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1d2e941a:::obj186:head) 2026-02-11T23:31:17.162+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1d2e941a:::obj186:head) v3 finish latency 0.000011 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.162+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.162+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.162+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.162+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2300 con 0x3752000 2026-02-11T23:31:17.163+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1dafe4f0:::obj47:head)]) -- 0x4d67980 con 0x3752000 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.163+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 224 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d44d00 con 0x3752000 2026-02-11T23:31:17.163+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.163+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.163+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.163+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 225 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1dafe4f0:::obj47:head) ==== 1061+0+0 (unknown 0 0 0) 0x4d78100 con 0x3752000 2026-02-11T23:31:17.163+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.163+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1dafe4f0:::obj47:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.163+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1dafe4f0:::obj47:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.163+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.163+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.163+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.163+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.163+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.163+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1dafe4f0:::obj47:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.163+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1e53d0d0:::obj151:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.163+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1e53d0d0:::obj151:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.163+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1e53d0d0:::obj151:head 2026-02-11T23:31:17.163+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1e53d0d0:::obj151:head v 107'151 2026-02-11T23:31:17.163+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.163+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.163+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.163+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1dafe4f0:::obj47:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.163+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1dafe4f0:::obj47:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 110/110 lb 1:1dafe4f0:::obj47:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.163+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.163+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1dafe4f0:::obj47:head) v3 2026-02-11T23:31:17.163+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1d2e941a:::obj186:head local-lis/les=109/110 n=93 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1dafe4f0:::obj47:head) 2026-02-11T23:31:17.163+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.163+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1dafe4f0:::obj47:head) v3 finish latency 0.000011 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.163+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.163+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.163+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2000 con 0x3752000 2026-02-11T23:31:17.163+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1e53d0d0:::obj151:head)]) -- 0x4d67b00 con 0x3752000 2026-02-11T23:31:17.163+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.164+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 226 ==== MOSDPGPush(1.0 110/109 [PushOp(1:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d451e0 con 0x3752000 2026-02-11T23:31:17.164+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.164+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.164+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 227 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1e53d0d0:::obj151:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d78680 con 0x3752000 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.164+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1e53d0d0:::obj151:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.164+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1e53d0d0:::obj151:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 110/109 [PushOp(1:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1e53d0d0:::obj151:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:1efaad2b:::obj120:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:1efaad2b:::obj120:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:1efaad2b:::obj120:head 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:1efaad2b:::obj120:head v 107'120 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.164+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1e53d0d0:::obj151:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1e53d0d0:::obj151:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:1e53d0d0:::obj151:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1e53d0d0:::obj151:head) v3 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1dafe4f0:::obj47:head local-lis/les=109/110 n=94 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1e53d0d0:::obj151:head) 2026-02-11T23:31:17.164+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1e53d0d0:::obj151:head) v3 finish latency 0.000010 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.164+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2480 con 0x3752000 2026-02-11T23:31:17.164+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:1efaad2b:::obj120:head)]) -- 0x4d67e00 con 0x3752000 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.164+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 228 ==== MOSDPGPush(1.0 110/109 [PushOp(1:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d456c0 con 0x3752000 2026-02-11T23:31:17.164+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.164+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.164+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 229 ==== pg_backfill(progress 1.0 e 110/110 lb 1:1efaad2b:::obj120:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d78c00 con 0x3752000 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.164+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:1efaad2b:::obj120:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.164+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1efaad2b:::obj120:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1efaad2b:::obj120:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:20b87b8b:::obj369:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:20b87b8b:::obj369:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:20b87b8b:::obj369:head 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:20b87b8b:::obj369:head v 107'369 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.164+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.164+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:1efaad2b:::obj120:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1efaad2b:::obj120:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 110/110 lb 1:1efaad2b:::obj120:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:1efaad2b:::obj120:head) v3 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1e53d0d0:::obj151:head local-lis/les=109/110 n=95 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:1efaad2b:::obj120:head) 2026-02-11T23:31:17.164+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:1efaad2b:::obj120:head) v3 finish latency 0.000009 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.164+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.164+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.164+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2c00 con 0x3752000 2026-02-11T23:31:17.164+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:20b87b8b:::obj369:head)]) -- 0x4af2900 con 0x3752000 2026-02-11T23:31:17.164+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.165+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 230 ==== MOSDPGPush(1.0 110/109 [PushOp(1:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4d45ba0 con 0x3752000 2026-02-11T23:31:17.165+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.165+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.165+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.165+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 231 ==== pg_backfill(progress 1.0 e 110/110 lb 1:20b87b8b:::obj369:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d79180 con 0x3752000 2026-02-11T23:31:17.165+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.165+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.165+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:20b87b8b:::obj369:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 110 2026-02-11T23:31:17.165+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.165+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:20b87b8b:::obj369:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.165+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.165+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:20b87b8b:::obj369:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.165+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.165+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.165+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.165+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.165+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:212e6483:::obj209:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.165+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:212e6483:::obj209:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.165+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:212e6483:::obj209:head 2026-02-11T23:31:17.165+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:212e6483:::obj209:head v 107'209 2026-02-11T23:31:17.165+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.165+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.165+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.165+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.165+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.165+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:20b87b8b:::obj369:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.165+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:20b87b8b:::obj369:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 110/110 lb 1:20b87b8b:::obj369:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.165+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.165+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:20b87b8b:::obj369:head) v3 2026-02-11T23:31:17.165+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:1efaad2b:::obj120:head local-lis/les=109/110 n=96 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:20b87b8b:::obj369:head) 2026-02-11T23:31:17.165+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.165+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:20b87b8b:::obj369:head) v3 finish latency 0.000015 2026-02-11T23:31:17.165+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.165+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.165+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.165+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.165+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.165+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.165+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.165+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3800 con 0x3752000 2026-02-11T23:31:17.165+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:212e6483:::obj209:head)]) -- 0x4c22900 con 0x3752000 2026-02-11T23:31:17.165+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.166+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 232 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e981a0 con 0x3752000 2026-02-11T23:31:17.166+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.166+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.166+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 233 ==== pg_backfill(progress 1.0 e 110/110 lb 1:212e6483:::obj209:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d79700 con 0x3752000 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.166+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:212e6483:::obj209:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.166+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:212e6483:::obj209:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:212e6483:::obj209:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2222704f:::obj161:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2222704f:::obj161:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2222704f:::obj161:head 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2222704f:::obj161:head v 107'161 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.166+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:212e6483:::obj209:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:212e6483:::obj209:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 110/110 lb 1:212e6483:::obj209:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:212e6483:::obj209:head) v3 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:20b87b8b:::obj369:head local-lis/les=109/110 n=97 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:212e6483:::obj209:head) 2026-02-11T23:31:17.166+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:212e6483:::obj209:head) v3 finish latency 0.000010 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.166+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4300 con 0x3752000 2026-02-11T23:31:17.166+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2222704f:::obj161:head)]) -- 0x4c23380 con 0x3752000 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.166+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 234 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e98680 con 0x3752000 2026-02-11T23:31:17.166+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.166+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.166+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 235 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2222704f:::obj161:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ea6000 con 0x3752000 2026-02-11T23:31:17.166+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2222704f:::obj161:head) v3 prio 15 type 95 cost 0 latency 0.000016 epoch 110 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 110/109 [PushOp(1:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.166+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2222704f:::obj161:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2222704f:::obj161:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2260a46a:::obj167:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2260a46a:::obj167:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2260a46a:::obj167:head 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2260a46a:::obj167:head v 107'167 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.166+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.166+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2222704f:::obj161:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2222704f:::obj161:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 110/110 lb 1:2222704f:::obj161:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2222704f:::obj161:head) v3 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:212e6483:::obj209:head local-lis/les=109/110 n=98 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2222704f:::obj161:head) 2026-02-11T23:31:17.166+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2222704f:::obj161:head) v3 finish latency 0.000014 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.166+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.166+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.166+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4900 con 0x3752000 2026-02-11T23:31:17.166+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2260a46a:::obj167:head)]) -- 0x4c23980 con 0x3752000 2026-02-11T23:31:17.166+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.167+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 236 ==== MOSDPGPush(1.0 110/109 [PushOp(1:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x4e98b60 con 0x3752000 2026-02-11T23:31:17.167+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 110 2026-02-11T23:31:17.167+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.167+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.167+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 237 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2260a46a:::obj167:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ea6580 con 0x3752000 2026-02-11T23:31:17.167+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.167+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.167+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2260a46a:::obj167:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 110 2026-02-11T23:31:17.167+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2260a46a:::obj167:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.167+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.167+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.167+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2260a46a:::obj167:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.167+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.167+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.167+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.167+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.167+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:233a42c1:::obj6:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.167+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:233a42c1:::obj6:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.167+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:233a42c1:::obj6:head 2026-02-11T23:31:17.167+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:233a42c1:::obj6:head v 107'6 2026-02-11T23:31:17.167+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.167+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.167+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.167+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.167+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.167+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2260a46a:::obj167:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.167+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2260a46a:::obj167:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 110/110 lb 1:2260a46a:::obj167:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.167+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.167+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2260a46a:::obj167:head) v3 2026-02-11T23:31:17.167+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2222704f:::obj161:head local-lis/les=109/110 n=99 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2260a46a:::obj167:head) 2026-02-11T23:31:17.167+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.167+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2260a46a:::obj167:head) v3 finish latency 0.000010 2026-02-11T23:31:17.167+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.167+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.167+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.167+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.167+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.167+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.167+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.167+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb4d80 con 0x3752000 2026-02-11T23:31:17.167+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:233a42c1:::obj6:head)]) -- 0x4eaa180 con 0x3752000 2026-02-11T23:31:17.167+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.167+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 238 ==== MOSDPGPush(1.0 110/109 [PushOp(1:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e99040 con 0x3752000 2026-02-11T23:31:17.167+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.167+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.168+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 239 ==== pg_backfill(progress 1.0 e 110/110 lb 1:233a42c1:::obj6:head) ==== 1060+0+0 (unknown 0 0 0) 0x4ea6b00 con 0x3752000 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.168+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:233a42c1:::obj6:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.168+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:233a42c1:::obj6:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:233a42c1:::obj6:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:239f544e:::obj128:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:239f544e:::obj128:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:239f544e:::obj128:head 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:239f544e:::obj128:head v 107'128 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.168+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:233a42c1:::obj6:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:233a42c1:::obj6:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 110/110 lb 1:233a42c1:::obj6:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:233a42c1:::obj6:head) v3 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2260a46a:::obj167:head local-lis/les=109/110 n=100 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:233a42c1:::obj6:head) 2026-02-11T23:31:17.168+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:233a42c1:::obj6:head) v3 finish latency 0.000019 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.168+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4cb5080 con 0x3752000 2026-02-11T23:31:17.168+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:239f544e:::obj128:head)]) -- 0x4eaa300 con 0x3752000 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.168+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 240 ==== MOSDPGPush(1.0 110/109 [PushOp(1:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4e99520 con 0x3752000 2026-02-11T23:31:17.168+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.168+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.168+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 241 ==== pg_backfill(progress 1.0 e 110/110 lb 1:239f544e:::obj128:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ea7080 con 0x3752000 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.168+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:239f544e:::obj128:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.168+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:239f544e:::obj128:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:239f544e:::obj128:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:242c40e0:::obj23:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:242c40e0:::obj23:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:242c40e0:::obj23:head 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:242c40e0:::obj23:head v 107'23 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.168+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.168+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:239f544e:::obj128:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:239f544e:::obj128:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 110/110 lb 1:239f544e:::obj128:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:239f544e:::obj128:head) v3 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:233a42c1:::obj6:head local-lis/les=109/110 n=101 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:239f544e:::obj128:head) 2026-02-11T23:31:17.168+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:239f544e:::obj128:head) v3 finish latency 0.000015 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.168+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.168+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.168+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3c80 con 0x3752000 2026-02-11T23:31:17.168+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:242c40e0:::obj23:head)]) -- 0x4eaa600 con 0x3752000 2026-02-11T23:31:17.168+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.169+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 242 ==== MOSDPGPush(1.0 110/109 [PushOp(1:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e99a00 con 0x3752000 2026-02-11T23:31:17.169+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.169+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.169+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 243 ==== pg_backfill(progress 1.0 e 110/110 lb 1:242c40e0:::obj23:head) ==== 1061+0+0 (unknown 0 0 0) 0x4ea7600 con 0x3752000 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.169+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:242c40e0:::obj23:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.169+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:242c40e0:::obj23:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000022 MOSDPGPush(1.0 110/109 [PushOp(1:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:242c40e0:::obj23:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:249a23dc:::obj422:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:249a23dc:::obj422:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:249a23dc:::obj422:head 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:249a23dc:::obj422:head v 107'422 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.169+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:242c40e0:::obj23:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.169+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:242c40e0:::obj23:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:242c40e0:::obj23:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.169+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.169+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:242c40e0:::obj23:head) v3 2026-02-11T23:31:17.169+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:239f544e:::obj128:head local-lis/les=109/110 n=102 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:242c40e0:::obj23:head) 2026-02-11T23:31:17.169+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.169+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:242c40e0:::obj23:head) v3 finish latency 0.000019 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.169+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.169+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3980 con 0x3752000 2026-02-11T23:31:17.169+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:249a23dc:::obj422:head)]) -- 0x4eaa780 con 0x3752000 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.169+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 244 ==== MOSDPGPush(1.0 110/109 [PushOp(1:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4a74000 con 0x3752000 2026-02-11T23:31:17.169+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.169+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.169+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 245 ==== pg_backfill(progress 1.0 e 110/110 lb 1:249a23dc:::obj422:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ea7b80 con 0x3752000 2026-02-11T23:31:17.169+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:249a23dc:::obj422:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.169+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:249a23dc:::obj422:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000022 MOSDPGPush(1.0 110/109 [PushOp(1:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.169+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:249a23dc:::obj422:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:24b5611b:::obj20:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.169+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:24b5611b:::obj20:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:24b5611b:::obj20:head 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:24b5611b:::obj20:head v 107'20 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.169+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.169+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.170+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:249a23dc:::obj422:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:249a23dc:::obj422:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 110/110 lb 1:249a23dc:::obj422:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:249a23dc:::obj422:head) v3 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:242c40e0:::obj23:head local-lis/les=109/110 n=103 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:249a23dc:::obj422:head) 2026-02-11T23:31:17.170+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:249a23dc:::obj422:head) v3 finish latency 0.000019 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.170+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.170+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2d80 con 0x3752000 2026-02-11T23:31:17.170+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:24b5611b:::obj20:head)]) -- 0x4eaaa80 con 0x3752000 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.170+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 246 ==== MOSDPGPush(1.0 110/109 [PushOp(1:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4a744e0 con 0x3752000 2026-02-11T23:31:17.170+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.170+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.170+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.170+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 247 ==== pg_backfill(progress 1.0 e 110/110 lb 1:24b5611b:::obj20:head) ==== 1061+0+0 (unknown 0 0 0) 0x4ea8100 con 0x3752000 2026-02-11T23:31:17.170+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.170+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:24b5611b:::obj20:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.170+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:24b5611b:::obj20:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.170+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.170+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.170+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.170+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.170+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.170+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:24b5611b:::obj20:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.170+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:24fc6e92:::obj17:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.170+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:24fc6e92:::obj17:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.170+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:24fc6e92:::obj17:head 2026-02-11T23:31:17.170+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:24fc6e92:::obj17:head v 107'17 2026-02-11T23:31:17.170+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.170+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.170+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.170+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:24b5611b:::obj20:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:24b5611b:::obj20:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 110/110 lb 1:24b5611b:::obj20:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:24b5611b:::obj20:head) v3 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:249a23dc:::obj422:head local-lis/les=109/110 n=104 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:24b5611b:::obj20:head) 2026-02-11T23:31:17.170+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:24b5611b:::obj20:head) v3 finish latency 0.000012 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.170+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.170+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.170+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3e00 con 0x3752000 2026-02-11T23:31:17.170+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:24fc6e92:::obj17:head)]) -- 0x4eaac00 con 0x3752000 2026-02-11T23:31:17.170+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:17.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:17.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:47.171563+0000) 2026-02-11T23:31:17.171+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 248 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4a749c0 con 0x3752000 2026-02-11T23:31:17.171+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.171+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.171+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 249 ==== pg_backfill(progress 1.0 e 110/110 lb 1:24fc6e92:::obj17:head) ==== 1061+0+0 (unknown 0 0 0) 0x4ea8680 con 0x3752000 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.171+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:24fc6e92:::obj17:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.171+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:24fc6e92:::obj17:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:24fc6e92:::obj17:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2540f982:::obj403:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2540f982:::obj403:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2540f982:::obj403:head 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2540f982:::obj403:head v 107'403 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.171+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:24fc6e92:::obj17:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.171+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:24fc6e92:::obj17:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:24fc6e92:::obj17:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.171+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.171+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:24fc6e92:::obj17:head) v3 2026-02-11T23:31:17.171+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24b5611b:::obj20:head local-lis/les=109/110 n=105 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:24fc6e92:::obj17:head) 2026-02-11T23:31:17.171+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.171+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:24fc6e92:::obj17:head) v3 finish latency 0.000015 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.171+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.171+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3080 con 0x3752000 2026-02-11T23:31:17.171+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2540f982:::obj403:head)]) -- 0x4eaad80 con 0x3752000 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.171+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 250 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4a74ea0 con 0x3752000 2026-02-11T23:31:17.171+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.171+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.171+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 251 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2540f982:::obj403:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ea8c00 con 0x3752000 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.171+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2540f982:::obj403:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.171+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2540f982:::obj403:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 110/109 [PushOp(1:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2540f982:::obj403:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2551b5e8:::obj231:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2551b5e8:::obj231:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2551b5e8:::obj231:head 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2551b5e8:::obj231:head v 107'231 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.171+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.171+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.171+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2540f982:::obj403:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.171+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2540f982:::obj403:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:2540f982:::obj403:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.171+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.171+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2540f982:::obj403:head) v3 2026-02-11T23:31:17.171+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:24fc6e92:::obj17:head local-lis/les=109/110 n=106 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2540f982:::obj403:head) 2026-02-11T23:31:17.171+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.171+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2540f982:::obj403:head) v3 finish latency 0.000023 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.171+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.172+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.172+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.172+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2f00 con 0x3752000 2026-02-11T23:31:17.172+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2551b5e8:::obj231:head)]) -- 0x4eab080 con 0x3752000 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.172+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 252 ==== MOSDPGPush(1.0 110/109 [PushOp(1:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4a75380 con 0x3752000 2026-02-11T23:31:17.172+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.172+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.172+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.172+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 253 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2551b5e8:::obj231:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ea9180 con 0x3752000 2026-02-11T23:31:17.172+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2551b5e8:::obj231:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.172+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.172+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2551b5e8:::obj231:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.172+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.172+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2551b5e8:::obj231:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.172+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 110/109 [PushOp(1:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.172+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.172+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.172+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.172+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:26b5008c:::obj336:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.172+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:26b5008c:::obj336:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.172+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:26b5008c:::obj336:head 2026-02-11T23:31:17.172+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:26b5008c:::obj336:head v 107'336 2026-02-11T23:31:17.172+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.172+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.172+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.172+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2551b5e8:::obj231:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.172+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2551b5e8:::obj231:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 110/110 lb 1:2551b5e8:::obj231:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.172+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.172+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2551b5e8:::obj231:head) v3 2026-02-11T23:31:17.172+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2540f982:::obj403:head local-lis/les=109/110 n=107 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2551b5e8:::obj231:head) 2026-02-11T23:31:17.172+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.172+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2551b5e8:::obj231:head) v3 finish latency 0.000012 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.172+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.172+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.172+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4af2900 con 0x3752000 2026-02-11T23:31:17.172+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:26b5008c:::obj336:head)]) -- 0x4eab380 con 0x3752000 2026-02-11T23:31:17.172+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.173+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 254 ==== MOSDPGPush(1.0 110/109 [PushOp(1:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4a75860 con 0x3752000 2026-02-11T23:31:17.173+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.173+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.173+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 255 ==== pg_backfill(progress 1.0 e 110/110 lb 1:26b5008c:::obj336:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ea9700 con 0x3752000 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.173+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:26b5008c:::obj336:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 110 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.173+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:26b5008c:::obj336:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:26b5008c:::obj336:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 110/109 [PushOp(1:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:26cf9b2d:::obj546:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:26cf9b2d:::obj546:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:26cf9b2d:::obj546:head 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:26cf9b2d:::obj546:head v 107'546 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.173+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:26b5008c:::obj336:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:26b5008c:::obj336:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 110/110 lb 1:26b5008c:::obj336:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:26b5008c:::obj336:head) v3 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2551b5e8:::obj231:head local-lis/les=109/110 n=108 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:26b5008c:::obj336:head) 2026-02-11T23:31:17.173+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:26b5008c:::obj336:head) v3 finish latency 0.000011 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.173+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d66600 con 0x3752000 2026-02-11T23:31:17.173+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:26cf9b2d:::obj546:head)]) -- 0x4eab680 con 0x3752000 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.173+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 256 ==== MOSDPGPush(1.0 110/109 [PushOp(1:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4a75d40 con 0x3752000 2026-02-11T23:31:17.173+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 110 2026-02-11T23:31:17.173+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.173+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 257 ==== pg_backfill(progress 1.0 e 110/110 lb 1:26cf9b2d:::obj546:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ed2000 con 0x3752000 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.173+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:26cf9b2d:::obj546:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 110 2026-02-11T23:31:17.173+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:26cf9b2d:::obj546:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:26cf9b2d:::obj546:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:27794e2d:::obj115:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:27794e2d:::obj115:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:27794e2d:::obj115:head 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:27794e2d:::obj115:head v 107'115 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.173+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.173+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:26cf9b2d:::obj546:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:26cf9b2d:::obj546:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 110/110 lb 1:26cf9b2d:::obj546:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:26cf9b2d:::obj546:head) v3 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26b5008c:::obj336:head local-lis/les=109/110 n=109 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:26cf9b2d:::obj546:head) 2026-02-11T23:31:17.173+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:26cf9b2d:::obj546:head) v3 finish latency 0.000010 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.173+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.173+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.173+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d66900 con 0x3752000 2026-02-11T23:31:17.173+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:27794e2d:::obj115:head)]) -- 0x4eab980 con 0x3752000 2026-02-11T23:31:17.173+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.174+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 258 ==== MOSDPGPush(1.0 110/109 [PushOp(1:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ed0340 con 0x3752000 2026-02-11T23:31:17.174+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.174+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.174+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 259 ==== pg_backfill(progress 1.0 e 110/110 lb 1:27794e2d:::obj115:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ed2580 con 0x3752000 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.174+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:27794e2d:::obj115:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.174+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:27794e2d:::obj115:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 110/109 [PushOp(1:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:27794e2d:::obj115:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:27952899:::obj201:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:27952899:::obj201:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:27952899:::obj201:head 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:27952899:::obj201:head v 107'201 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.174+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:27794e2d:::obj115:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.174+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:27794e2d:::obj115:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 110/110 lb 1:27794e2d:::obj115:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.174+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.174+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:27794e2d:::obj115:head) v3 2026-02-11T23:31:17.174+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:26cf9b2d:::obj546:head local-lis/les=109/110 n=110 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:27794e2d:::obj115:head) 2026-02-11T23:31:17.174+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.174+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:27794e2d:::obj115:head) v3 finish latency 0.000011 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.174+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.174+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d66c00 con 0x3752000 2026-02-11T23:31:17.174+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:27952899:::obj201:head)]) -- 0x4eabc80 con 0x3752000 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.174+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 260 ==== MOSDPGPush(1.0 110/109 [PushOp(1:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ed0820 con 0x3752000 2026-02-11T23:31:17.174+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.174+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.174+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 261 ==== pg_backfill(progress 1.0 e 110/110 lb 1:27952899:::obj201:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ed2b00 con 0x3752000 2026-02-11T23:31:17.174+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:27952899:::obj201:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.174+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:27952899:::obj201:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 110/109 [PushOp(1:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:27952899:::obj201:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:29cd40a5:::obj549:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:29cd40a5:::obj549:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:29cd40a5:::obj549:head 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:29cd40a5:::obj549:head v 107'549 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.174+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.174+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.174+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:27952899:::obj201:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.174+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:27952899:::obj201:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 110/110 lb 1:27952899:::obj201:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.174+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.174+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:27952899:::obj201:head) v3 2026-02-11T23:31:17.174+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27794e2d:::obj115:head local-lis/les=109/110 n=111 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:27952899:::obj201:head) 2026-02-11T23:31:17.174+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.174+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:27952899:::obj201:head) v3 finish latency 0.000010 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.174+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.175+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.175+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.175+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d66f00 con 0x3752000 2026-02-11T23:31:17.175+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:29cd40a5:::obj549:head)]) -- 0x4cb4300 con 0x3752000 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.175+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 262 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ed0d00 con 0x3752000 2026-02-11T23:31:17.175+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.175+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.175+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.175+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 263 ==== pg_backfill(progress 1.0 e 110/110 lb 1:29cd40a5:::obj549:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ed3080 con 0x3752000 2026-02-11T23:31:17.175+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.175+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:29cd40a5:::obj549:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.175+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.175+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:29cd40a5:::obj549:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.175+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.175+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 110/109 [PushOp(1:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.175+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.175+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:29cd40a5:::obj549:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.175+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.175+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2a627378:::obj100:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.175+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2a627378:::obj100:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.175+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2a627378:::obj100:head 2026-02-11T23:31:17.175+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2a627378:::obj100:head v 107'100 2026-02-11T23:31:17.175+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.175+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.175+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.175+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:29cd40a5:::obj549:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.175+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:29cd40a5:::obj549:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 110/110 lb 1:29cd40a5:::obj549:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.175+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.175+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:29cd40a5:::obj549:head) v3 2026-02-11T23:31:17.175+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:27952899:::obj201:head local-lis/les=109/110 n=112 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:29cd40a5:::obj549:head) 2026-02-11T23:31:17.175+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.175+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:29cd40a5:::obj549:head) v3 finish latency 0.000021 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.175+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.175+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.175+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d67200 con 0x3752000 2026-02-11T23:31:17.175+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2a627378:::obj100:head)]) -- 0x4c23e00 con 0x3752000 2026-02-11T23:31:17.175+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.176+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 264 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ed11e0 con 0x3752000 2026-02-11T23:31:17.176+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.176+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.176+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 265 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2a627378:::obj100:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ed3600 con 0x3752000 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.176+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2a627378:::obj100:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.176+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2a627378:::obj100:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 110/109 [PushOp(1:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2a627378:::obj100:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2acce685:::obj583:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2acce685:::obj583:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2acce685:::obj583:head 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2acce685:::obj583:head v 107'583 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.176+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2a627378:::obj100:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2a627378:::obj100:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 110/110 lb 1:2a627378:::obj100:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2a627378:::obj100:head) v3 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:29cd40a5:::obj549:head local-lis/les=109/110 n=113 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2a627378:::obj100:head) 2026-02-11T23:31:17.176+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2a627378:::obj100:head) v3 finish latency 0.000016 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.176+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d67500 con 0x3752000 2026-02-11T23:31:17.176+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2acce685:::obj583:head)]) -- 0x4c23380 con 0x3752000 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.176+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 266 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ed16c0 con 0x3752000 2026-02-11T23:31:17.176+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.176+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.176+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 267 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2acce685:::obj583:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ed3b80 con 0x3752000 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.176+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2acce685:::obj583:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.176+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2acce685:::obj583:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(1.0 110/109 [PushOp(1:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2acce685:::obj583:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2b184b8d:::obj560:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2b184b8d:::obj560:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2b184b8d:::obj560:head 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2b184b8d:::obj560:head v 107'560 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.176+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.176+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2acce685:::obj583:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2acce685:::obj583:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 110/110 lb 1:2acce685:::obj583:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2acce685:::obj583:head) v3 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2a627378:::obj100:head local-lis/les=109/110 n=114 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2acce685:::obj583:head) 2026-02-11T23:31:17.176+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2acce685:::obj583:head) v3 finish latency 0.000014 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.176+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.176+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.176+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d67800 con 0x3752000 2026-02-11T23:31:17.176+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2b184b8d:::obj560:head)]) -- 0x4ba2300 con 0x3752000 2026-02-11T23:31:17.176+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.177+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 268 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ed1ba0 con 0x3752000 2026-02-11T23:31:17.177+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 110 2026-02-11T23:31:17.177+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.177+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 269 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2b184b8d:::obj560:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ed4100 con 0x3752000 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.177+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2b184b8d:::obj560:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.177+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2b184b8d:::obj560:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2b184b8d:::obj560:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2c8d8a76:::obj467:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2c8d8a76:::obj467:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2c8d8a76:::obj467:head 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2c8d8a76:::obj467:head v 107'467 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.177+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2b184b8d:::obj560:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.177+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2b184b8d:::obj560:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 110/110 lb 1:2b184b8d:::obj560:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.177+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.177+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2b184b8d:::obj560:head) v3 2026-02-11T23:31:17.177+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2acce685:::obj583:head local-lis/les=109/110 n=115 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2b184b8d:::obj560:head) 2026-02-11T23:31:17.177+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.177+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2b184b8d:::obj560:head) v3 finish latency 0.000015 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.177+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.177+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d66180 con 0x3752000 2026-02-11T23:31:17.177+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2c8d8a76:::obj467:head)]) -- 0x4ba2480 con 0x3752000 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.177+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 270 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ee61a0 con 0x3752000 2026-02-11T23:31:17.177+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.177+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.177+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 271 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2c8d8a76:::obj467:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ed4680 con 0x3752000 2026-02-11T23:31:17.177+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2c8d8a76:::obj467:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.177+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2c8d8a76:::obj467:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2c8d8a76:::obj467:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2caf4871:::obj405:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2caf4871:::obj405:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2caf4871:::obj405:head 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2caf4871:::obj405:head v 107'405 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.177+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.177+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.177+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2c8d8a76:::obj467:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.177+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2c8d8a76:::obj467:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 110/110 lb 1:2c8d8a76:::obj467:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.177+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.177+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2c8d8a76:::obj467:head) v3 2026-02-11T23:31:17.177+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2b184b8d:::obj560:head local-lis/les=109/110 n=116 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2c8d8a76:::obj467:head) 2026-02-11T23:31:17.177+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.177+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2c8d8a76:::obj467:head) v3 finish latency 0.000015 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.177+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.178+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.178+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.178+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d67980 con 0x3752000 2026-02-11T23:31:17.178+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2caf4871:::obj405:head)]) -- 0x4ba3800 con 0x3752000 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.178+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 272 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ee6680 con 0x3752000 2026-02-11T23:31:17.178+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.178+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.178+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.178+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 273 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2caf4871:::obj405:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ed4c00 con 0x3752000 2026-02-11T23:31:17.178+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.178+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.178+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.178+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2caf4871:::obj405:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 110 2026-02-11T23:31:17.178+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2caf4871:::obj405:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2caf4871:::obj405:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.178+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 110/109 [PushOp(1:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.178+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.178+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.178+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.178+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2d59679a:::obj179:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.178+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2d59679a:::obj179:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.178+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2d59679a:::obj179:head 2026-02-11T23:31:17.178+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2d59679a:::obj179:head v 107'179 2026-02-11T23:31:17.178+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.178+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.178+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.178+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2caf4871:::obj405:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.178+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2caf4871:::obj405:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 110/110 lb 1:2caf4871:::obj405:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.178+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.178+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2caf4871:::obj405:head) v3 2026-02-11T23:31:17.178+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2c8d8a76:::obj467:head local-lis/les=109/110 n=117 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2caf4871:::obj405:head) 2026-02-11T23:31:17.178+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.178+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2caf4871:::obj405:head) v3 finish latency 0.000011 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.178+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.178+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.178+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d67b00 con 0x3752000 2026-02-11T23:31:17.178+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2d59679a:::obj179:head)]) -- 0x4cb5080 con 0x3752000 2026-02-11T23:31:17.178+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.179+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 274 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ee6b60 con 0x3752000 2026-02-11T23:31:17.179+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.179+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.179+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 275 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2d59679a:::obj179:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ed5180 con 0x3752000 2026-02-11T23:31:17.179+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2d59679a:::obj179:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.179+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2d59679a:::obj179:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2d59679a:::obj179:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 110/109 [PushOp(1:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2df8f997:::obj267:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2df8f997:::obj267:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2df8f997:::obj267:head 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2df8f997:::obj267:head v 107'267 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.179+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2d59679a:::obj179:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2d59679a:::obj179:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 110/110 lb 1:2d59679a:::obj179:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2d59679a:::obj179:head) v3 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2caf4871:::obj405:head local-lis/les=109/110 n=118 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2d59679a:::obj179:head) 2026-02-11T23:31:17.179+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2d59679a:::obj179:head) v3 finish latency 0.000010 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.179+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d67e00 con 0x3752000 2026-02-11T23:31:17.179+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2df8f997:::obj267:head)]) -- 0x4eaa300 con 0x3752000 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.179+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 276 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ee7040 con 0x3752000 2026-02-11T23:31:17.179+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 110 2026-02-11T23:31:17.179+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.179+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 277 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2df8f997:::obj267:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ed5700 con 0x3752000 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.179+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2df8f997:::obj267:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.179+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2df8f997:::obj267:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 110/109 [PushOp(1:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2df8f997:::obj267:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2e5f3a17:::obj520:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2e5f3a17:::obj520:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2e5f3a17:::obj520:head 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2e5f3a17:::obj520:head v 107'520 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.179+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.179+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2df8f997:::obj267:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2df8f997:::obj267:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:2df8f997:::obj267:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2df8f997:::obj267:head) v3 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2d59679a:::obj179:head local-lis/les=109/110 n=119 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2df8f997:::obj267:head) 2026-02-11T23:31:17.179+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2df8f997:::obj267:head) v3 finish latency 0.000015 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.179+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.179+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.179+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d66600 con 0x3752000 2026-02-11T23:31:17.179+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2e5f3a17:::obj520:head)]) -- 0x4eaa780 con 0x3752000 2026-02-11T23:31:17.179+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.180+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 278 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ee7520 con 0x3752000 2026-02-11T23:31:17.180+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.180+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.180+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 279 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2e5f3a17:::obj520:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ef2000 con 0x3752000 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.180+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2e5f3a17:::obj520:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 110 2026-02-11T23:31:17.180+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2e5f3a17:::obj520:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2e5f3a17:::obj520:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 110/109 [PushOp(1:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2e69a8fc:::obj598:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2e69a8fc:::obj598:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2e69a8fc:::obj598:head 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2e69a8fc:::obj598:head v 107'598 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.180+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2e5f3a17:::obj520:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.180+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2e5f3a17:::obj520:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 110/110 lb 1:2e5f3a17:::obj520:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.180+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.180+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2e5f3a17:::obj520:head) v3 2026-02-11T23:31:17.180+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2df8f997:::obj267:head local-lis/les=109/110 n=120 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2e5f3a17:::obj520:head) 2026-02-11T23:31:17.180+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.180+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2e5f3a17:::obj520:head) v3 finish latency 0.000010 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.180+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.180+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d66900 con 0x3752000 2026-02-11T23:31:17.180+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2e69a8fc:::obj598:head)]) -- 0x4eaac00 con 0x3752000 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.180+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 280 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4ee7a00 con 0x3752000 2026-02-11T23:31:17.180+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 110 2026-02-11T23:31:17.180+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.180+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 281 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2e69a8fc:::obj598:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ef2580 con 0x3752000 2026-02-11T23:31:17.180+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2e69a8fc:::obj598:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.180+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2e69a8fc:::obj598:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.180+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2e69a8fc:::obj598:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 110/109 [PushOp(1:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2e72dfbc:::obj232:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.180+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2e72dfbc:::obj232:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2e72dfbc:::obj232:head 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2e72dfbc:::obj232:head v 107'232 2026-02-11T23:31:17.180+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.180+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.181+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.181+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2e69a8fc:::obj598:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2e69a8fc:::obj598:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 110/110 lb 1:2e69a8fc:::obj598:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2e69a8fc:::obj598:head) v3 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e5f3a17:::obj520:head local-lis/les=109/110 n=121 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2e69a8fc:::obj598:head) 2026-02-11T23:31:17.181+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2e69a8fc:::obj598:head) v3 finish latency 0.000011 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.181+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.181+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d66c00 con 0x3752000 2026-02-11T23:31:17.181+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2e72dfbc:::obj232:head)]) -- 0x4eab080 con 0x3752000 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.181+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 282 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x448e680 con 0x3752000 2026-02-11T23:31:17.181+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.181+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.181+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.181+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 283 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2e72dfbc:::obj232:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ef2b00 con 0x3752000 2026-02-11T23:31:17.181+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.181+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2e72dfbc:::obj232:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.181+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.181+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2e72dfbc:::obj232:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.181+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.181+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.181+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.181+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.181+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2e72dfbc:::obj232:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.181+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2eeda62f:::obj154:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.181+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2eeda62f:::obj154:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.181+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2eeda62f:::obj154:head 2026-02-11T23:31:17.181+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2eeda62f:::obj154:head v 107'154 2026-02-11T23:31:17.181+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.181+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.181+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.181+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2e72dfbc:::obj232:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2e72dfbc:::obj232:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:2e72dfbc:::obj232:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2e72dfbc:::obj232:head) v3 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e69a8fc:::obj598:head local-lis/les=109/110 n=122 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2e72dfbc:::obj232:head) 2026-02-11T23:31:17.181+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2e72dfbc:::obj232:head) v3 finish latency 0.000016 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.181+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.181+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.181+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d66f00 con 0x3752000 2026-02-11T23:31:17.181+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2eeda62f:::obj154:head)]) -- 0x4eab680 con 0x3752000 2026-02-11T23:31:17.181+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.182+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 284 ==== MOSDPGPush(1.0 110/109 [PushOp(1:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x448eb60 con 0x3752000 2026-02-11T23:31:17.182+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.182+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.182+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 285 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2eeda62f:::obj154:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ef3080 con 0x3752000 2026-02-11T23:31:17.182+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2eeda62f:::obj154:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.182+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2eeda62f:::obj154:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 110/109 [PushOp(1:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2eeda62f:::obj154:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:2f478301:::obj479:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:2f478301:::obj479:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:2f478301:::obj479:head 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:2f478301:::obj479:head v 107'479 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.182+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2eeda62f:::obj154:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.182+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2eeda62f:::obj154:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 110/110 lb 1:2eeda62f:::obj154:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.182+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.182+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2eeda62f:::obj154:head) v3 2026-02-11T23:31:17.182+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2e72dfbc:::obj232:head local-lis/les=109/110 n=123 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2eeda62f:::obj154:head) 2026-02-11T23:31:17.182+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.182+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2eeda62f:::obj154:head) v3 finish latency 0.000011 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.182+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.182+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d67200 con 0x3752000 2026-02-11T23:31:17.182+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:2f478301:::obj479:head)]) -- 0x4eabc80 con 0x3752000 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.182+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 286 ==== MOSDPGPush(1.0 110/109 [PushOp(1:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x448f040 con 0x3752000 2026-02-11T23:31:17.182+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 110 2026-02-11T23:31:17.182+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.182+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 287 ==== pg_backfill(progress 1.0 e 110/110 lb 1:2f478301:::obj479:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ef3600 con 0x3752000 2026-02-11T23:31:17.182+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:2f478301:::obj479:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.182+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2f478301:::obj479:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 110/109 [PushOp(1:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2f478301:::obj479:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:303427ce:::obj485:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:303427ce:::obj485:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:303427ce:::obj485:head 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:303427ce:::obj485:head v 107'485 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.182+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.182+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.182+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:2f478301:::obj479:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.182+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2f478301:::obj479:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 110/110 lb 1:2f478301:::obj479:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.182+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.182+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:2f478301:::obj479:head) v3 2026-02-11T23:31:17.182+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2eeda62f:::obj154:head local-lis/les=109/110 n=124 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:2f478301:::obj479:head) 2026-02-11T23:31:17.182+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.182+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:2f478301:::obj479:head) v3 finish latency 0.000019 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.182+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.183+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.183+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.183+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4d67500 con 0x3752000 2026-02-11T23:31:17.183+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:303427ce:::obj485:head)]) -- 0x4ba3c80 con 0x3752000 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.183+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 288 ==== MOSDPGPush(1.0 110/109 [PushOp(1:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38f44e0 con 0x3752000 2026-02-11T23:31:17.183+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 110 2026-02-11T23:31:17.183+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.183+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.183+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.183+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 289 ==== pg_backfill(progress 1.0 e 110/110 lb 1:303427ce:::obj485:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ef3b80 con 0x3752000 2026-02-11T23:31:17.183+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:303427ce:::obj485:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.183+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.183+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:303427ce:::obj485:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.183+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.183+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 110/109 [PushOp(1:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.183+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.183+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:303427ce:::obj485:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.183+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.183+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:320b4068:::obj138:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.183+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:320b4068:::obj138:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.183+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:320b4068:::obj138:head 2026-02-11T23:31:17.183+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:320b4068:::obj138:head v 107'138 2026-02-11T23:31:17.183+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.183+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.183+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:17.183+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:303427ce:::obj485:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.183+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:303427ce:::obj485:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 1.0 e 110/110 lb 1:303427ce:::obj485:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.183+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.183+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:303427ce:::obj485:head) v3 2026-02-11T23:31:17.183+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:2f478301:::obj479:head local-lis/les=109/110 n=125 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:303427ce:::obj485:head) 2026-02-11T23:31:17.183+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.183+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:303427ce:::obj485:head) v3 finish latency 0.000020 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.183+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.183+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.183+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4c23980 con 0x3752000 2026-02-11T23:31:17.183+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:320b4068:::obj138:head)]) -- 0x4ba2d80 con 0x3752000 2026-02-11T23:31:17.183+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.184+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 290 ==== MOSDPGPush(1.0 110/109 [PushOp(1:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3921d40 con 0x3752000 2026-02-11T23:31:17.184+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 110 2026-02-11T23:31:17.184+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.184+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 291 ==== pg_backfill(progress 1.0 e 110/110 lb 1:320b4068:::obj138:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ef4100 con 0x3752000 2026-02-11T23:31:17.184+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:320b4068:::obj138:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.184+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:320b4068:::obj138:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.184+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 110/109 [PushOp(1:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.184+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.184+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.184+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:320b4068:::obj138:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.184+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.184+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.184+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:325eabfb:::obj310:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:325eabfb:::obj310:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.184+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:325eabfb:::obj310:head 2026-02-11T23:31:17.184+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:325eabfb:::obj310:head v 107'310 2026-02-11T23:31:17.184+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.184+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.184+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.184+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.184+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:320b4068:::obj138:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.184+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:320b4068:::obj138:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 110/110 lb 1:320b4068:::obj138:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.184+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.184+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:320b4068:::obj138:head) v3 2026-02-11T23:31:17.184+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:303427ce:::obj485:head local-lis/les=109/110 n=126 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:320b4068:::obj138:head) 2026-02-11T23:31:17.184+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.184+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:320b4068:::obj138:head) v3 finish latency 0.000022 2026-02-11T23:31:17.184+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.184+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.184+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.184+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4910000 con 0x3752000 2026-02-11T23:31:17.184+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:325eabfb:::obj310:head)]) -- 0x4c23e00 con 0x3752000 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.184+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 292 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38f2340 con 0x3752000 2026-02-11T23:31:17.184+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.184+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.184+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.184+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 293 ==== pg_backfill(progress 1.0 e 110/110 lb 1:325eabfb:::obj310:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ef4680 con 0x3752000 2026-02-11T23:31:17.184+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:325eabfb:::obj310:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.184+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.184+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:325eabfb:::obj310:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.184+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.184+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.184+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:325eabfb:::obj310:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.184+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 110/109 [PushOp(1:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.184+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.184+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.184+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.185+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3263b485:::obj202:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.185+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3263b485:::obj202:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.185+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3263b485:::obj202:head 2026-02-11T23:31:17.185+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3263b485:::obj202:head v 107'202 2026-02-11T23:31:17.185+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.185+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.185+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.185+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:325eabfb:::obj310:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:325eabfb:::obj310:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 110/110 lb 1:325eabfb:::obj310:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:325eabfb:::obj310:head) v3 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:320b4068:::obj138:head local-lis/les=109/110 n=127 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:325eabfb:::obj310:head) 2026-02-11T23:31:17.185+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:325eabfb:::obj310:head) v3 finish latency 0.000016 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.185+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.185+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4910180 con 0x3752000 2026-02-11T23:31:17.185+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3263b485:::obj202:head)]) -- 0x4ba3080 con 0x3752000 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.185+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 294 ==== MOSDPGPush(1.0 110/109 [PushOp(1:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x391e000 con 0x3752000 2026-02-11T23:31:17.185+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 110 2026-02-11T23:31:17.185+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.185+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 295 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3263b485:::obj202:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ef4c00 con 0x3752000 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.185+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3263b485:::obj202:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.185+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3263b485:::obj202:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.185+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 110/109 [PushOp(1:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.185+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.185+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3263b485:::obj202:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:326c82ab:::obj527:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:326c82ab:::obj527:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:326c82ab:::obj527:head 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:326c82ab:::obj527:head v 107'527 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.185+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.185+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.185+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3263b485:::obj202:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.185+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3263b485:::obj202:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 110/110 lb 1:3263b485:::obj202:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.185+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.185+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3263b485:::obj202:head) v3 2026-02-11T23:31:17.185+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:325eabfb:::obj310:head local-lis/les=109/110 n=128 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3263b485:::obj202:head) 2026-02-11T23:31:17.185+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.185+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3263b485:::obj202:head) v3 finish latency 0.000019 2026-02-11T23:31:17.185+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.185+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.185+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.185+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4910600 con 0x3752000 2026-02-11T23:31:17.185+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:326c82ab:::obj527:head)]) -- 0x4910300 con 0x3752000 2026-02-11T23:31:17.185+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.186+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 296 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x391e4e0 con 0x3752000 2026-02-11T23:31:17.186+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 110 2026-02-11T23:31:17.186+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.186+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 297 ==== pg_backfill(progress 1.0 e 110/110 lb 1:326c82ab:::obj527:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ef5180 con 0x3752000 2026-02-11T23:31:17.186+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:326c82ab:::obj527:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.186+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:326c82ab:::obj527:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 110/109 [PushOp(1:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:326c82ab:::obj527:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3286c571:::obj350:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3286c571:::obj350:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3286c571:::obj350:head 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3286c571:::obj350:head v 107'350 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.186+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:326c82ab:::obj527:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:326c82ab:::obj527:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 110/110 lb 1:326c82ab:::obj527:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:326c82ab:::obj527:head) v3 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3263b485:::obj202:head local-lis/les=109/110 n=129 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:326c82ab:::obj527:head) 2026-02-11T23:31:17.186+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:326c82ab:::obj527:head) v3 finish latency 0.000020 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.186+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4910a80 con 0x3752000 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.186+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3286c571:::obj350:head)]) -- 0x4910780 con 0x3752000 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.186+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 298 ==== MOSDPGPush(1.0 110/109 [PushOp(1:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x391e820 con 0x3752000 2026-02-11T23:31:17.186+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.186+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.186+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 299 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3286c571:::obj350:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ef5700 con 0x3752000 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.186+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3286c571:::obj350:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.186+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3286c571:::obj350:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3286c571:::obj350:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:32db648a:::obj568:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:32db648a:::obj568:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:32db648a:::obj568:head 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:32db648a:::obj568:head v 107'568 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.186+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.186+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.186+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3286c571:::obj350:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3286c571:::obj350:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 110/110 lb 1:3286c571:::obj350:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3286c571:::obj350:head) v3 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:326c82ab:::obj527:head local-lis/les=109/110 n=130 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3286c571:::obj350:head) 2026-02-11T23:31:17.186+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.186+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3286c571:::obj350:head) v3 finish latency 0.000019 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.186+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.187+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.187+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.187+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4910c00 con 0x3752000 2026-02-11T23:31:17.187+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:32db648a:::obj568:head)]) -- 0x4ba2300 con 0x3752000 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.187+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 300 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3975a00 con 0x3752000 2026-02-11T23:31:17.187+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.187+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.187+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.187+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 301 ==== pg_backfill(progress 1.0 e 110/110 lb 1:32db648a:::obj568:head) ==== 1062+0+0 (unknown 0 0 0) 0x4920000 con 0x3752000 2026-02-11T23:31:17.187+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.187+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:32db648a:::obj568:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.187+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:32db648a:::obj568:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.187+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.187+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.187+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 110/109 [PushOp(1:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.187+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:32db648a:::obj568:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.187+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.187+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.187+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3372ec01:::obj225:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.187+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3372ec01:::obj225:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.187+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3372ec01:::obj225:head 2026-02-11T23:31:17.187+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3372ec01:::obj225:head v 107'225 2026-02-11T23:31:17.187+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.187+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.187+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.187+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:32db648a:::obj568:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.187+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:32db648a:::obj568:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 110/110 lb 1:32db648a:::obj568:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.187+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.187+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:32db648a:::obj568:head) v3 2026-02-11T23:31:17.187+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3286c571:::obj350:head local-lis/les=109/110 n=131 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:32db648a:::obj568:head) 2026-02-11T23:31:17.187+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.187+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:32db648a:::obj568:head) v3 finish latency 0.000017 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.187+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.187+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.187+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4910d80 con 0x3752000 2026-02-11T23:31:17.187+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3372ec01:::obj225:head)]) -- 0x4c23980 con 0x3752000 2026-02-11T23:31:17.187+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.188+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 302 ==== MOSDPGPush(1.0 110/109 [PushOp(1:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3974ea0 con 0x3752000 2026-02-11T23:31:17.188+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.188+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.188+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 303 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3372ec01:::obj225:head) ==== 1062+0+0 (unknown 0 0 0) 0x4920580 con 0x3752000 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.188+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3372ec01:::obj225:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.188+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3372ec01:::obj225:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 110/109 [PushOp(1:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3372ec01:::obj225:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:339fd60b:::obj93:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:339fd60b:::obj93:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:339fd60b:::obj93:head 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:339fd60b:::obj93:head v 107'93 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.188+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3372ec01:::obj225:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.188+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3372ec01:::obj225:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 110/110 lb 1:3372ec01:::obj225:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.188+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.188+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3372ec01:::obj225:head) v3 2026-02-11T23:31:17.188+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:32db648a:::obj568:head local-lis/les=109/110 n=132 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3372ec01:::obj225:head) 2026-02-11T23:31:17.188+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.188+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3372ec01:::obj225:head) v3 finish latency 0.000016 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.188+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.188+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4910f00 con 0x3752000 2026-02-11T23:31:17.188+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:339fd60b:::obj93:head)]) -- 0x4d67800 con 0x3752000 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.188+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 304 ==== MOSDPGPush(1.0 110/109 [PushOp(1:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3975380 con 0x3752000 2026-02-11T23:31:17.188+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.188+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.188+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 305 ==== pg_backfill(progress 1.0 e 110/110 lb 1:339fd60b:::obj93:head) ==== 1061+0+0 (unknown 0 0 0) 0x4920b00 con 0x3752000 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.188+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:339fd60b:::obj93:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.188+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:339fd60b:::obj93:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:339fd60b:::obj93:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:33a4b6a3:::obj360:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:33a4b6a3:::obj360:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:33a4b6a3:::obj360:head 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:33a4b6a3:::obj360:head v 107'360 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.188+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.188+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.188+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:339fd60b:::obj93:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.188+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:339fd60b:::obj93:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 110/110 lb 1:339fd60b:::obj93:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.188+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.188+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:339fd60b:::obj93:head) v3 2026-02-11T23:31:17.188+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3372ec01:::obj225:head local-lis/les=109/110 n=133 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:339fd60b:::obj93:head) 2026-02-11T23:31:17.188+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.188+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:339fd60b:::obj93:head) v3 finish latency 0.000017 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.188+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.189+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.189+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.189+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4911080 con 0x3752000 2026-02-11T23:31:17.189+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:33a4b6a3:::obj360:head)]) -- 0x4d66180 con 0x3752000 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.189+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 306 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x391ed00 con 0x3752000 2026-02-11T23:31:17.189+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.189+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.189+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.189+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 307 ==== pg_backfill(progress 1.0 e 110/110 lb 1:33a4b6a3:::obj360:head) ==== 1062+0+0 (unknown 0 0 0) 0x4921080 con 0x3752000 2026-02-11T23:31:17.189+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.189+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:33a4b6a3:::obj360:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.189+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.189+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:33a4b6a3:::obj360:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.189+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.189+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 110/109 [PushOp(1:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.189+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:33a4b6a3:::obj360:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.189+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.189+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.189+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3421eea3:::obj166:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.189+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3421eea3:::obj166:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.189+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3421eea3:::obj166:head 2026-02-11T23:31:17.189+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3421eea3:::obj166:head v 107'166 2026-02-11T23:31:17.189+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.189+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.189+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:17.189+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:33a4b6a3:::obj360:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.189+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:33a4b6a3:::obj360:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 110/110 lb 1:33a4b6a3:::obj360:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.189+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.189+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:33a4b6a3:::obj360:head) v3 2026-02-11T23:31:17.189+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:339fd60b:::obj93:head local-lis/les=109/110 n=134 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:33a4b6a3:::obj360:head) 2026-02-11T23:31:17.189+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.189+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:33a4b6a3:::obj360:head) v3 finish latency 0.000016 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.189+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.189+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.189+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4911200 con 0x3752000 2026-02-11T23:31:17.189+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3421eea3:::obj166:head)]) -- 0x4d67b00 con 0x3752000 2026-02-11T23:31:17.189+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.190+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 308 ==== MOSDPGPush(1.0 110/109 [PushOp(1:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x391f1e0 con 0x3752000 2026-02-11T23:31:17.190+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.190+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.190+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 309 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3421eea3:::obj166:head) ==== 1062+0+0 (unknown 0 0 0) 0x4921600 con 0x3752000 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.190+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3421eea3:::obj166:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.190+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3421eea3:::obj166:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 110/109 [PushOp(1:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3421eea3:::obj166:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:360ee05a:::obj519:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:360ee05a:::obj519:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:360ee05a:::obj519:head 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:360ee05a:::obj519:head v 107'519 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.190+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3421eea3:::obj166:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3421eea3:::obj166:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 110/110 lb 1:3421eea3:::obj166:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3421eea3:::obj166:head) v3 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:33a4b6a3:::obj360:head local-lis/les=109/110 n=135 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3421eea3:::obj166:head) 2026-02-11T23:31:17.190+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3421eea3:::obj166:head) v3 finish latency 0.000017 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.190+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4911380 con 0x3752000 2026-02-11T23:31:17.190+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:360ee05a:::obj519:head)]) -- 0x4d66600 con 0x3752000 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.190+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 310 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x391f6c0 con 0x3752000 2026-02-11T23:31:17.190+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.190+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.190+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 311 ==== pg_backfill(progress 1.0 e 110/110 lb 1:360ee05a:::obj519:head) ==== 1062+0+0 (unknown 0 0 0) 0x4921b80 con 0x3752000 2026-02-11T23:31:17.190+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:360ee05a:::obj519:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 110 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.190+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:360ee05a:::obj519:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 110/109 [PushOp(1:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:360ee05a:::obj519:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3655ac21:::obj70:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3655ac21:::obj70:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3655ac21:::obj70:head 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3655ac21:::obj70:head v 107'70 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.190+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.190+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:360ee05a:::obj519:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:360ee05a:::obj519:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 110/110 lb 1:360ee05a:::obj519:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:360ee05a:::obj519:head) v3 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3421eea3:::obj166:head local-lis/les=109/110 n=136 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:360ee05a:::obj519:head) 2026-02-11T23:31:17.190+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:360ee05a:::obj519:head) v3 finish latency 0.000015 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.190+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.190+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.190+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4911500 con 0x3752000 2026-02-11T23:31:17.190+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3655ac21:::obj70:head)]) -- 0x4d66c00 con 0x3752000 2026-02-11T23:31:17.190+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.191+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 312 ==== MOSDPGPush(1.0 110/109 [PushOp(1:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x391fba0 con 0x3752000 2026-02-11T23:31:17.191+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 110 2026-02-11T23:31:17.191+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.191+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.191+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 313 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3655ac21:::obj70:head) ==== 1061+0+0 (unknown 0 0 0) 0x4922100 con 0x3752000 2026-02-11T23:31:17.191+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.191+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3655ac21:::obj70:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.191+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3655ac21:::obj70:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.191+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.191+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.191+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.191+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 110/109 [PushOp(1:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.191+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.191+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.191+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.191+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3655ac21:::obj70:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.191+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.191+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:372fd24d:::obj442:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.191+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:372fd24d:::obj442:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.191+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:372fd24d:::obj442:head 2026-02-11T23:31:17.191+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:372fd24d:::obj442:head v 107'442 2026-02-11T23:31:17.191+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.191+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.191+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.191+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.191+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.191+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3655ac21:::obj70:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.191+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3655ac21:::obj70:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 110/110 lb 1:3655ac21:::obj70:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.191+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.191+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3655ac21:::obj70:head) v3 2026-02-11T23:31:17.191+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:360ee05a:::obj519:head local-lis/les=109/110 n=137 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3655ac21:::obj70:head) 2026-02-11T23:31:17.191+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.191+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3655ac21:::obj70:head) v3 finish latency 0.000017 2026-02-11T23:31:17.191+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.191+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.191+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.191+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.191+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.191+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.191+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.191+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4911680 con 0x3752000 2026-02-11T23:31:17.191+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:372fd24d:::obj442:head)]) -- 0x4d66f00 con 0x3752000 2026-02-11T23:31:17.191+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.192+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 314 ==== MOSDPGPush(1.0 110/109 [PushOp(1:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x391e9c0 con 0x3752000 2026-02-11T23:31:17.192+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 110 2026-02-11T23:31:17.192+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.192+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 315 ==== pg_backfill(progress 1.0 e 110/110 lb 1:372fd24d:::obj442:head) ==== 1062+0+0 (unknown 0 0 0) 0x4922680 con 0x3752000 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 110/109 [PushOp(1:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.192+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:372fd24d:::obj442:head) v3 prio 15 type 95 cost 0 latency 0.000022 epoch 110 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.192+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:372fd24d:::obj442:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:37ff9db7:::obj453:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:37ff9db7:::obj453:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:37ff9db7:::obj453:head 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:372fd24d:::obj442:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:37ff9db7:::obj453:head v 107'453 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.192+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:372fd24d:::obj442:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.192+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:372fd24d:::obj442:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 1.0 e 110/110 lb 1:372fd24d:::obj442:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.192+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.192+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:372fd24d:::obj442:head) v3 2026-02-11T23:31:17.192+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3655ac21:::obj70:head local-lis/les=109/110 n=138 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:372fd24d:::obj442:head) 2026-02-11T23:31:17.192+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.192+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:372fd24d:::obj442:head) v3 finish latency 0.000027 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.192+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.192+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4911800 con 0x3752000 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.192+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:37ff9db7:::obj453:head)]) -- 0x4d67500 con 0x3752000 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.192+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 316 ==== MOSDPGPush(1.0 110/109 [PushOp(1:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39216c0 con 0x3752000 2026-02-11T23:31:17.192+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 110 2026-02-11T23:31:17.192+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.192+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 317 ==== pg_backfill(progress 1.0 e 110/110 lb 1:37ff9db7:::obj453:head) ==== 1062+0+0 (unknown 0 0 0) 0x4922c00 con 0x3752000 2026-02-11T23:31:17.192+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:37ff9db7:::obj453:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.192+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:37ff9db7:::obj453:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 110/109 [PushOp(1:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:37ff9db7:::obj453:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:384eca0c:::obj484:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:384eca0c:::obj484:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:384eca0c:::obj484:head 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:384eca0c:::obj484:head v 107'484 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.192+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.192+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.192+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:37ff9db7:::obj453:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.192+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:37ff9db7:::obj453:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 110/110 lb 1:37ff9db7:::obj453:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.192+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.192+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:37ff9db7:::obj453:head) v3 2026-02-11T23:31:17.192+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:372fd24d:::obj442:head local-lis/les=109/110 n=139 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:37ff9db7:::obj453:head) 2026-02-11T23:31:17.192+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.192+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:37ff9db7:::obj453:head) v3 finish latency 0.000023 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.192+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.193+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.193+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.193+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4911980 con 0x3752000 2026-02-11T23:31:17.193+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:384eca0c:::obj484:head)]) -- 0x4eaa780 con 0x3752000 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.193+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.193+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 318 ==== MOSDPGPush(1.0 110/109 [PushOp(1:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3921ba0 con 0x3752000 2026-02-11T23:31:17.193+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.193+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.193+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.193+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 319 ==== pg_backfill(progress 1.0 e 110/110 lb 1:384eca0c:::obj484:head) ==== 1062+0+0 (unknown 0 0 0) 0x4923180 con 0x3752000 2026-02-11T23:31:17.193+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.193+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:384eca0c:::obj484:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 110 2026-02-11T23:31:17.193+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:384eca0c:::obj484:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.193+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.193+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.193+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.193+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.193+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:384eca0c:::obj484:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.193+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.193+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:385733f0:::obj251:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.193+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:385733f0:::obj251:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.193+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:385733f0:::obj251:head 2026-02-11T23:31:17.193+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:385733f0:::obj251:head v 107'251 2026-02-11T23:31:17.193+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.193+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.193+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.193+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:384eca0c:::obj484:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.193+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:384eca0c:::obj484:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 110/110 lb 1:384eca0c:::obj484:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.193+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.193+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:384eca0c:::obj484:head) v3 2026-02-11T23:31:17.193+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:37ff9db7:::obj453:head local-lis/les=109/110 n=140 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:384eca0c:::obj484:head) 2026-02-11T23:31:17.193+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.193+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:384eca0c:::obj484:head) v3 finish latency 0.000033 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.193+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.193+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.193+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4911b00 con 0x3752000 2026-02-11T23:31:17.193+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:385733f0:::obj251:head)]) -- 0x4eab080 con 0x3752000 2026-02-11T23:31:17.193+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.194+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 320 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3920d00 con 0x3752000 2026-02-11T23:31:17.194+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.194+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.194+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.194+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 321 ==== pg_backfill(progress 1.0 e 110/110 lb 1:385733f0:::obj251:head) ==== 1062+0+0 (unknown 0 0 0) 0x4923700 con 0x3752000 2026-02-11T23:31:17.194+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:385733f0:::obj251:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.194+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.194+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:385733f0:::obj251:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.194+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.194+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 110/109 [PushOp(1:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:385733f0:::obj251:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3945af93:::obj271:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.194+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3945af93:::obj271:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3945af93:::obj271:head 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3945af93:::obj271:head v 107'271 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.194+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.194+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:385733f0:::obj251:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:385733f0:::obj251:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 110/110 lb 1:385733f0:::obj251:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:385733f0:::obj251:head) v3 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:384eca0c:::obj484:head local-lis/les=109/110 n=141 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:385733f0:::obj251:head) 2026-02-11T23:31:17.194+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:385733f0:::obj251:head) v3 finish latency 0.000023 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.194+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.194+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4911c80 con 0x3752000 2026-02-11T23:31:17.194+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3945af93:::obj271:head)]) -- 0x4eabc80 con 0x3752000 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.194+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 322 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x39201a0 con 0x3752000 2026-02-11T23:31:17.194+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.194+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.194+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 323 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3945af93:::obj271:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a6c000 con 0x3752000 2026-02-11T23:31:17.194+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3945af93:::obj271:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.194+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3945af93:::obj271:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.194+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.194+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3945af93:::obj271:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3a764ab4:::obj37:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3a764ab4:::obj37:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3a764ab4:::obj37:head 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3a764ab4:::obj37:head v 107'37 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.194+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.194+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.194+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.194+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.194+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3945af93:::obj271:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3945af93:::obj271:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 110/110 lb 1:3945af93:::obj271:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3945af93:::obj271:head) v3 2026-02-11T23:31:17.194+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:385733f0:::obj251:head local-lis/les=109/110 n=142 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3945af93:::obj271:head) 2026-02-11T23:31:17.194+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.195+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3945af93:::obj271:head) v3 finish latency 0.000034 2026-02-11T23:31:17.195+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.195+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.195+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.195+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4910180 con 0x3752000 2026-02-11T23:31:17.195+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3a764ab4:::obj37:head)]) -- 0x4910000 con 0x3752000 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.195+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.195+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 324 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3920680 con 0x3752000 2026-02-11T23:31:17.195+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.195+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.195+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 325 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3a764ab4:::obj37:head) ==== 1061+0+0 (unknown 0 0 0) 0x4a6c580 con 0x3752000 2026-02-11T23:31:17.195+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3a764ab4:::obj37:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.195+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3a764ab4:::obj37:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.195+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 110/109 [PushOp(1:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.195+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.195+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.195+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3a764ab4:::obj37:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.195+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.195+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.195+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3b0aa4c9:::obj511:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3b0aa4c9:::obj511:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.195+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3b0aa4c9:::obj511:head 2026-02-11T23:31:17.195+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3b0aa4c9:::obj511:head v 107'511 2026-02-11T23:31:17.195+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.195+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.195+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.195+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.195+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3a764ab4:::obj37:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.195+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3a764ab4:::obj37:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 1.0 e 110/110 lb 1:3a764ab4:::obj37:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.195+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.195+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3a764ab4:::obj37:head) v3 2026-02-11T23:31:17.195+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3945af93:::obj271:head local-lis/les=109/110 n=143 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3a764ab4:::obj37:head) 2026-02-11T23:31:17.195+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.195+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3a764ab4:::obj37:head) v3 finish latency 0.000023 2026-02-11T23:31:17.195+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.195+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.195+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.195+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4911e00 con 0x3752000 2026-02-11T23:31:17.195+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3b0aa4c9:::obj511:head)]) -- 0x4910600 con 0x3752000 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.195+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.195+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.196+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 326 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38f3860 con 0x3752000 2026-02-11T23:31:17.196+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.196+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.196+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 327 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3b0aa4c9:::obj511:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a6cb00 con 0x3752000 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.196+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3b0aa4c9:::obj511:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.196+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3b0aa4c9:::obj511:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 110/109 [PushOp(1:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3b0aa4c9:::obj511:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3bc05e5f:::obj343:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3bc05e5f:::obj343:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3bc05e5f:::obj343:head 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3bc05e5f:::obj343:head v 107'343 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.196+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3b0aa4c9:::obj511:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3b0aa4c9:::obj511:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 1.0 e 110/110 lb 1:3b0aa4c9:::obj511:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3b0aa4c9:::obj511:head) v3 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3a764ab4:::obj37:head local-lis/les=109/110 n=144 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3b0aa4c9:::obj511:head) 2026-02-11T23:31:17.196+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3b0aa4c9:::obj511:head) v3 finish latency 0.000018 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.196+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3c80 con 0x3752000 2026-02-11T23:31:17.196+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3bc05e5f:::obj343:head)]) -- 0x4ba2480 con 0x3752000 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.196+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 328 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38f3d40 con 0x3752000 2026-02-11T23:31:17.196+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.196+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.196+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 329 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3bc05e5f:::obj343:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a6d080 con 0x3752000 2026-02-11T23:31:17.196+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3bc05e5f:::obj343:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.196+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3bc05e5f:::obj343:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 110/109 [PushOp(1:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3bc05e5f:::obj343:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3cb06661:::obj109:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3cb06661:::obj109:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3cb06661:::obj109:head 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3cb06661:::obj109:head v 107'109 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.196+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.196+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.196+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3bc05e5f:::obj343:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3bc05e5f:::obj343:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 110/110 lb 1:3bc05e5f:::obj343:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3bc05e5f:::obj343:head) v3 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3b0aa4c9:::obj511:head local-lis/les=109/110 n=145 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3bc05e5f:::obj343:head) 2026-02-11T23:31:17.196+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.196+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3bc05e5f:::obj343:head) v3 finish latency 0.000016 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.196+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.197+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.197+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.197+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba2d80 con 0x3752000 2026-02-11T23:31:17.197+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3cb06661:::obj109:head)]) -- 0x4954000 con 0x3752000 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.197+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 330 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x38f2b60 con 0x3752000 2026-02-11T23:31:17.197+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.197+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.197+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.197+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 331 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3cb06661:::obj109:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a6d600 con 0x3752000 2026-02-11T23:31:17.197+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.197+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3cb06661:::obj109:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 110 2026-02-11T23:31:17.197+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3cb06661:::obj109:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.197+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.197+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.197+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.197+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3cb06661:::obj109:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.197+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.197+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.197+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3cb2c87d:::obj96:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.197+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3cb2c87d:::obj96:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.197+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3cb2c87d:::obj96:head 2026-02-11T23:31:17.197+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3cb2c87d:::obj96:head v 107'96 2026-02-11T23:31:17.197+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.197+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.197+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.197+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3cb06661:::obj109:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.197+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3cb06661:::obj109:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 110/110 lb 1:3cb06661:::obj109:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.197+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.197+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3cb06661:::obj109:head) v3 2026-02-11T23:31:17.197+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3bc05e5f:::obj343:head local-lis/les=109/110 n=146 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3cb06661:::obj109:head) 2026-02-11T23:31:17.197+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.197+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3cb06661:::obj109:head) v3 finish latency 0.000017 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.197+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.197+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.197+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4ba3080 con 0x3752000 2026-02-11T23:31:17.197+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3cb2c87d:::obj96:head)]) -- 0x4954300 con 0x3752000 2026-02-11T23:31:17.197+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.198+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 332 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38f3040 con 0x3752000 2026-02-11T23:31:17.198+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.198+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.198+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 333 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3cb2c87d:::obj96:head) ==== 1061+0+0 (unknown 0 0 0) 0x4a6db80 con 0x3752000 2026-02-11T23:31:17.198+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3cb2c87d:::obj96:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.198+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3cb2c87d:::obj96:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.198+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.198+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3cb2c87d:::obj96:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3cc20580:::obj177:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3cc20580:::obj177:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3cc20580:::obj177:head 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3cc20580:::obj177:head v 107'177 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.198+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.198+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.198+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3cb2c87d:::obj96:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.198+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3cb2c87d:::obj96:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 110/110 lb 1:3cb2c87d:::obj96:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.198+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.198+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3cb2c87d:::obj96:head) v3 2026-02-11T23:31:17.198+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb06661:::obj109:head local-lis/les=109/110 n=147 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3cb2c87d:::obj96:head) 2026-02-11T23:31:17.198+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.198+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3cb2c87d:::obj96:head) v3 finish latency 0.000017 2026-02-11T23:31:17.198+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.198+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.198+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4916300 con 0x3752000 2026-02-11T23:31:17.198+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3cc20580:::obj177:head)]) -- 0x4916000 con 0x3752000 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.198+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 334 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38f3520 con 0x3752000 2026-02-11T23:31:17.198+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.198+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.198+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 335 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3cc20580:::obj177:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a6e100 con 0x3752000 2026-02-11T23:31:17.198+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3cc20580:::obj177:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 110 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.198+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3cc20580:::obj177:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.198+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.198+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3cc20580:::obj177:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(1.0 110/109 [PushOp(1:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3d7e2b4d:::obj143:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3d7e2b4d:::obj143:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3d7e2b4d:::obj143:head 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3d7e2b4d:::obj143:head v 107'143 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.198+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.198+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.198+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.198+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.198+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3cc20580:::obj177:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.198+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3cc20580:::obj177:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 1.0 e 110/110 lb 1:3cc20580:::obj177:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.198+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.198+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3cc20580:::obj177:head) v3 2026-02-11T23:31:17.198+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cb2c87d:::obj96:head local-lis/les=109/110 n=148 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3cc20580:::obj177:head) 2026-02-11T23:31:17.198+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.198+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3cc20580:::obj177:head) v3 finish latency 0.000015 2026-02-11T23:31:17.198+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.199+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.199+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.199+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4916780 con 0x3752000 2026-02-11T23:31:17.199+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3d7e2b4d:::obj143:head)]) -- 0x4916480 con 0x3752000 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.199+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 336 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x448fba0 con 0x3752000 2026-02-11T23:31:17.199+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.199+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.199+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 337 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3d7e2b4d:::obj143:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a6e680 con 0x3752000 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.199+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3d7e2b4d:::obj143:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 110 2026-02-11T23:31:17.199+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3d7e2b4d:::obj143:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.199+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.199+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.199+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.199+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.199+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.199+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.199+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3d7e2b4d:::obj143:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.199+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3d8750d8:::obj471:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3d8750d8:::obj471:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.199+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3d8750d8:::obj471:head 2026-02-11T23:31:17.199+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3d8750d8:::obj471:head v 107'471 2026-02-11T23:31:17.199+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.199+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.199+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.199+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.199+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3d7e2b4d:::obj143:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.199+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3d7e2b4d:::obj143:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 110/110 lb 1:3d7e2b4d:::obj143:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.199+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.199+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3d7e2b4d:::obj143:head) v3 2026-02-11T23:31:17.199+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3cc20580:::obj177:head local-lis/les=109/110 n=149 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3d7e2b4d:::obj143:head) 2026-02-11T23:31:17.199+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.199+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3d7e2b4d:::obj143:head) v3 finish latency 0.000017 2026-02-11T23:31:17.199+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.199+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.199+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.199+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4916c00 con 0x3752000 2026-02-11T23:31:17.199+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3d8750d8:::obj471:head)]) -- 0x4916900 con 0x3752000 2026-02-11T23:31:17.199+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.200+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 338 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x449c1a0 con 0x3752000 2026-02-11T23:31:17.200+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.200+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.200+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.200+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 339 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3d8750d8:::obj471:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a6ec00 con 0x3752000 2026-02-11T23:31:17.200+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3d8750d8:::obj471:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.200+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3d8750d8:::obj471:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.200+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.200+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.200+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.200+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 110/109 [PushOp(1:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.200+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.200+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.200+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3d8750d8:::obj471:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.200+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.200+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.200+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3d88a4f9:::obj551:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.200+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3d88a4f9:::obj551:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.200+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3d88a4f9:::obj551:head 2026-02-11T23:31:17.200+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3d88a4f9:::obj551:head v 107'551 2026-02-11T23:31:17.200+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.200+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.200+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:17.200+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.200+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.200+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3d8750d8:::obj471:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.200+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3d8750d8:::obj471:head) v3 prio 15 cost 0 latency 0.000099 pg_backfill(progress 1.0 e 110/110 lb 1:3d8750d8:::obj471:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.200+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.200+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3d8750d8:::obj471:head) v3 2026-02-11T23:31:17.200+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d7e2b4d:::obj143:head local-lis/les=109/110 n=150 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3d8750d8:::obj471:head) 2026-02-11T23:31:17.200+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.200+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3d8750d8:::obj471:head) v3 finish latency 0.000015 2026-02-11T23:31:17.200+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.200+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.200+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.200+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.200+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.200+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.200+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.200+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4916d80 con 0x3752000 2026-02-11T23:31:17.200+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3d88a4f9:::obj551:head)]) -- 0x4954480 con 0x3752000 2026-02-11T23:31:17.200+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.200+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 340 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x449c680 con 0x3752000 2026-02-11T23:31:17.200+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.200+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.200+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 341 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3d88a4f9:::obj551:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a6f180 con 0x3752000 2026-02-11T23:31:17.200+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3d88a4f9:::obj551:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.200+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.200+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3d88a4f9:::obj551:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.200+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.200+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.200+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 110/109 [PushOp(1:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3d88a4f9:::obj551:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3e07a120:::obj540:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.201+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3e07a120:::obj540:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3e07a120:::obj540:head 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3e07a120:::obj540:head v 107'540 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.201+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.201+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3d88a4f9:::obj551:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3d88a4f9:::obj551:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 110/110 lb 1:3d88a4f9:::obj551:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3d88a4f9:::obj551:head) v3 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d8750d8:::obj471:head local-lis/les=109/110 n=151 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3d88a4f9:::obj551:head) 2026-02-11T23:31:17.201+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3d88a4f9:::obj551:head) v3 finish latency 0.000020 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.201+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.201+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4916f00 con 0x3752000 2026-02-11T23:31:17.201+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3e07a120:::obj540:head)]) -- 0x4954780 con 0x3752000 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.201+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 342 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x448f6c0 con 0x3752000 2026-02-11T23:31:17.201+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.201+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.201+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 343 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3e07a120:::obj540:head) ==== 1062+0+0 (unknown 0 0 0) 0x4a6f700 con 0x3752000 2026-02-11T23:31:17.201+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3e07a120:::obj540:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.201+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.201+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3e07a120:::obj540:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.201+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.201+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.201+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3e07a120:::obj540:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3ed3e33d:::obj376:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.201+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3ed3e33d:::obj376:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3ed3e33d:::obj376:head 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3ed3e33d:::obj376:head v 107'376 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.201+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.201+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.201+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3e07a120:::obj540:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3e07a120:::obj540:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 110/110 lb 1:3e07a120:::obj540:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3e07a120:::obj540:head) v3 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3d88a4f9:::obj551:head local-lis/les=109/110 n=152 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3e07a120:::obj540:head) 2026-02-11T23:31:17.201+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3e07a120:::obj540:head) v3 finish latency 0.000011 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.201+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.201+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.201+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4917080 con 0x3752000 2026-02-11T23:31:17.201+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3ed3e33d:::obj376:head)]) -- 0x4954a80 con 0x3752000 2026-02-11T23:31:17.201+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.202+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 344 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x38f2680 con 0x3752000 2026-02-11T23:31:17.202+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.202+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.202+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.202+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 345 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3ed3e33d:::obj376:head) ==== 1062+0+0 (unknown 0 0 0) 0x4966000 con 0x3752000 2026-02-11T23:31:17.202+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.202+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3ed3e33d:::obj376:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.202+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3ed3e33d:::obj376:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.202+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.202+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 110/109 [PushOp(1:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3ed3e33d:::obj376:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3efbd745:::obj84:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.202+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3efbd745:::obj84:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3efbd745:::obj84:head 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3efbd745:::obj84:head v 107'84 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.202+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.202+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3ed3e33d:::obj376:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.202+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3ed3e33d:::obj376:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 110/110 lb 1:3ed3e33d:::obj376:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.202+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.202+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3ed3e33d:::obj376:head) v3 2026-02-11T23:31:17.202+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3e07a120:::obj540:head local-lis/les=109/110 n=153 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3ed3e33d:::obj376:head) 2026-02-11T23:31:17.202+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.202+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3ed3e33d:::obj376:head) v3 finish latency 0.000013 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.202+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.202+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.202+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4917200 con 0x3752000 2026-02-11T23:31:17.202+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3efbd745:::obj84:head)]) -- 0x4954d80 con 0x3752000 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.202+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 346 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x268aea0 con 0x3752000 2026-02-11T23:31:17.202+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.202+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.202+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 347 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3efbd745:::obj84:head) ==== 1061+0+0 (unknown 0 0 0) 0x4966580 con 0x3752000 2026-02-11T23:31:17.202+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3efbd745:::obj84:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.202+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.202+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3efbd745:::obj84:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.202+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.202+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.202+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 110/109 [PushOp(1:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.202+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3efbd745:::obj84:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.202+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.203+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3fc284d5:::obj140:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.203+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3fc284d5:::obj140:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.203+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3fc284d5:::obj140:head 2026-02-11T23:31:17.203+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3fc284d5:::obj140:head v 107'140 2026-02-11T23:31:17.203+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.203+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.203+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:17.203+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3efbd745:::obj84:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3efbd745:::obj84:head) v3 prio 15 cost 0 latency 0.000103 pg_backfill(progress 1.0 e 110/110 lb 1:3efbd745:::obj84:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3efbd745:::obj84:head) v3 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3ed3e33d:::obj376:head local-lis/les=109/110 n=154 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3efbd745:::obj84:head) 2026-02-11T23:31:17.203+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3efbd745:::obj84:head) v3 finish latency 0.000022 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.203+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.203+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4917380 con 0x3752000 2026-02-11T23:31:17.203+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3fc284d5:::obj140:head)]) -- 0x4954f00 con 0x3752000 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.203+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 348 ==== MOSDPGPush(1.0 110/109 [PushOp(1:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x268b380 con 0x3752000 2026-02-11T23:31:17.203+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.203+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.203+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 349 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3fc284d5:::obj140:head) ==== 1062+0+0 (unknown 0 0 0) 0x4966b00 con 0x3752000 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.203+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3fc284d5:::obj140:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.203+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3fc284d5:::obj140:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.203+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.203+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3fc284d5:::obj140:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:3fe0a5f5:::obj46:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:3fe0a5f5:::obj46:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:3fe0a5f5:::obj46:head 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:3fe0a5f5:::obj46:head v 107'46 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.203+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.203+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.203+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3fc284d5:::obj140:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.203+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3fc284d5:::obj140:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 110/110 lb 1:3fc284d5:::obj140:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.203+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.203+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3fc284d5:::obj140:head) v3 2026-02-11T23:31:17.203+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3efbd745:::obj84:head local-lis/les=109/110 n=155 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3fc284d5:::obj140:head) 2026-02-11T23:31:17.203+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.203+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3fc284d5:::obj140:head) v3 finish latency 0.000012 2026-02-11T23:31:17.203+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.203+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.203+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.203+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4917680 con 0x3752000 2026-02-11T23:31:17.203+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:3fe0a5f5:::obj46:head)]) -- 0x4917500 con 0x3752000 2026-02-11T23:31:17.203+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.204+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 350 ==== MOSDPGPush(1.0 110/109 [PushOp(1:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x268a4e0 con 0x3752000 2026-02-11T23:31:17.204+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.204+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.204+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 351 ==== pg_backfill(progress 1.0 e 110/110 lb 1:3fe0a5f5:::obj46:head) ==== 1061+0+0 (unknown 0 0 0) 0x4967080 con 0x3752000 2026-02-11T23:31:17.204+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:3fe0a5f5:::obj46:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.204+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3fe0a5f5:::obj46:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.204+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.204+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.204+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.204+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.204+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3fe0a5f5:::obj46:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.204+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.204+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:401e0147:::obj213:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:401e0147:::obj213:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.204+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:401e0147:::obj213:head 2026-02-11T23:31:17.204+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:401e0147:::obj213:head v 107'213 2026-02-11T23:31:17.204+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.204+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.204+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.204+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.204+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:3fe0a5f5:::obj46:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.204+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3fe0a5f5:::obj46:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 110/110 lb 1:3fe0a5f5:::obj46:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.204+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.204+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:3fe0a5f5:::obj46:head) v3 2026-02-11T23:31:17.204+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fc284d5:::obj140:head local-lis/les=109/110 n=156 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:3fe0a5f5:::obj46:head) 2026-02-11T23:31:17.204+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.204+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:3fe0a5f5:::obj46:head) v3 finish latency 0.000014 2026-02-11T23:31:17.204+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.204+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.204+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.204+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4917b00 con 0x3752000 2026-02-11T23:31:17.204+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:401e0147:::obj213:head)]) -- 0x4917800 con 0x3752000 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.204+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 352 ==== MOSDPGPush(1.0 110/109 [PushOp(1:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3968820 con 0x3752000 2026-02-11T23:31:17.204+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 110 2026-02-11T23:31:17.204+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.204+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 353 ==== pg_backfill(progress 1.0 e 110/110 lb 1:401e0147:::obj213:head) ==== 1062+0+0 (unknown 0 0 0) 0x4967600 con 0x3752000 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.205+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:401e0147:::obj213:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 110 2026-02-11T23:31:17.205+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:401e0147:::obj213:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.204+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 110/109 [PushOp(1:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:401e0147:::obj213:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:401e941b:::obj398:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:401e941b:::obj398:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:401e941b:::obj398:head 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:401e941b:::obj398:head v 107'398 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.205+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:401e0147:::obj213:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.205+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:401e0147:::obj213:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 1.0 e 110/110 lb 1:401e0147:::obj213:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.205+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.205+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:401e0147:::obj213:head) v3 2026-02-11T23:31:17.205+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:3fe0a5f5:::obj46:head local-lis/les=109/110 n=157 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:401e0147:::obj213:head) 2026-02-11T23:31:17.205+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.205+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:401e0147:::obj213:head) v3 finish latency 0.000017 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.205+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x497e000 con 0x3752000 2026-02-11T23:31:17.205+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:401e941b:::obj398:head)]) -- 0x4917c80 con 0x3752000 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.205+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 354 ==== MOSDPGPush(1.0 110/109 [PushOp(1:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x268b860 con 0x3752000 2026-02-11T23:31:17.205+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.205+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.205+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 355 ==== pg_backfill(progress 1.0 e 110/110 lb 1:401e941b:::obj398:head) ==== 1062+0+0 (unknown 0 0 0) 0x4967b80 con 0x3752000 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.205+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:401e941b:::obj398:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.205+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:401e941b:::obj398:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 110/109 [PushOp(1:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:401e941b:::obj398:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:40d1abdf:::obj254:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:40d1abdf:::obj254:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:40d1abdf:::obj254:head 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:40d1abdf:::obj254:head v 107'254 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.205+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:401e941b:::obj398:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.205+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:401e941b:::obj398:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 1.0 e 110/110 lb 1:401e941b:::obj398:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.205+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.205+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:401e941b:::obj398:head) v3 2026-02-11T23:31:17.205+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e0147:::obj213:head local-lis/les=109/110 n=158 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:401e941b:::obj398:head) 2026-02-11T23:31:17.205+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.205+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:401e941b:::obj398:head) v3 finish latency 0.000012 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.205+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.205+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.205+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x497e480 con 0x3752000 2026-02-11T23:31:17.205+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:40d1abdf:::obj254:head)]) -- 0x497e180 con 0x3752000 2026-02-11T23:31:17.205+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.206+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 356 ==== MOSDPGPush(1.0 110/109 [PushOp(1:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x268bd40 con 0x3752000 2026-02-11T23:31:17.206+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.206+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.206+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 357 ==== pg_backfill(progress 1.0 e 110/110 lb 1:40d1abdf:::obj254:head) ==== 1062+0+0 (unknown 0 0 0) 0x4968100 con 0x3752000 2026-02-11T23:31:17.206+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.206+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:40d1abdf:::obj254:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.206+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:40d1abdf:::obj254:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.206+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.206+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.206+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.206+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.206+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:40d1abdf:::obj254:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.206+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.206+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.206+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:41892471:::obj524:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.206+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:41892471:::obj524:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.206+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:41892471:::obj524:head 2026-02-11T23:31:17.206+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:41892471:::obj524:head v 107'524 2026-02-11T23:31:17.206+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.206+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.206+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.206+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:40d1abdf:::obj254:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.206+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:40d1abdf:::obj254:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 110/110 lb 1:40d1abdf:::obj254:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.206+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.206+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:40d1abdf:::obj254:head) v3 2026-02-11T23:31:17.206+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:401e941b:::obj398:head local-lis/les=109/110 n=159 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:40d1abdf:::obj254:head) 2026-02-11T23:31:17.206+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.206+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:40d1abdf:::obj254:head) v3 finish latency 0.000015 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.206+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.206+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.206+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x497e600 con 0x3752000 2026-02-11T23:31:17.206+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:41892471:::obj524:head)]) -- 0x4955200 con 0x3752000 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.206+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 358 ==== MOSDPGPush(1.0 110/109 [PushOp(1:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3968340 con 0x3752000 2026-02-11T23:31:17.206+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.206+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.206+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 359 ==== pg_backfill(progress 1.0 e 110/110 lb 1:41892471:::obj524:head) ==== 1062+0+0 (unknown 0 0 0) 0x4968680 con 0x3752000 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.206+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:41892471:::obj524:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.206+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:41892471:::obj524:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.206+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.206+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.206+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.206+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:41892471:::obj524:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.206+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.206+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.206+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:41ea52ba:::obj210:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.206+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:41ea52ba:::obj210:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:41ea52ba:::obj210:head 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:41ea52ba:::obj210:head v 107'210 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.207+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.207+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.207+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:41892471:::obj524:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.207+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:41892471:::obj524:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 110/110 lb 1:41892471:::obj524:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.207+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.207+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:41892471:::obj524:head) v3 2026-02-11T23:31:17.207+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:40d1abdf:::obj254:head local-lis/les=109/110 n=160 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:41892471:::obj524:head) 2026-02-11T23:31:17.207+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.207+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:41892471:::obj524:head) v3 finish latency 0.000012 2026-02-11T23:31:17.207+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.207+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.207+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x497ea80 con 0x3752000 2026-02-11T23:31:17.207+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:41ea52ba:::obj210:head)]) -- 0x497e780 con 0x3752000 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.207+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 360 ==== MOSDPGPush(1.0 110/109 [PushOp(1:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39691e0 con 0x3752000 2026-02-11T23:31:17.207+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.207+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.207+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 361 ==== pg_backfill(progress 1.0 e 110/110 lb 1:41ea52ba:::obj210:head) ==== 1062+0+0 (unknown 0 0 0) 0x4968c00 con 0x3752000 2026-02-11T23:31:17.207+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:41ea52ba:::obj210:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.207+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:41ea52ba:::obj210:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 110/109 [PushOp(1:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.207+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.207+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:41ea52ba:::obj210:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:424ce90f:::obj439:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:424ce90f:::obj439:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:424ce90f:::obj439:head 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:424ce90f:::obj439:head v 107'439 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.207+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.207+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.207+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:41ea52ba:::obj210:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.207+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:41ea52ba:::obj210:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 110/110 lb 1:41ea52ba:::obj210:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.207+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.207+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:41ea52ba:::obj210:head) v3 2026-02-11T23:31:17.207+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41892471:::obj524:head local-lis/les=109/110 n=161 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:41ea52ba:::obj210:head) 2026-02-11T23:31:17.207+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.207+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:41ea52ba:::obj210:head) v3 finish latency 0.000012 2026-02-11T23:31:17.207+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.207+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.207+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.207+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x497ef00 con 0x3752000 2026-02-11T23:31:17.207+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:424ce90f:::obj439:head)]) -- 0x497ec00 con 0x3752000 2026-02-11T23:31:17.207+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.208+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 362 ==== MOSDPGPush(1.0 110/109 [PushOp(1:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39696c0 con 0x3752000 2026-02-11T23:31:17.208+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.208+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.208+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 363 ==== pg_backfill(progress 1.0 e 110/110 lb 1:424ce90f:::obj439:head) ==== 1062+0+0 (unknown 0 0 0) 0x4969180 con 0x3752000 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.208+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:424ce90f:::obj439:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.208+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:424ce90f:::obj439:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:424ce90f:::obj439:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:42d0e5ef:::obj468:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:42d0e5ef:::obj468:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:42d0e5ef:::obj468:head 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:42d0e5ef:::obj468:head v 107'468 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.208+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:424ce90f:::obj439:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:424ce90f:::obj439:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 110/110 lb 1:424ce90f:::obj439:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:424ce90f:::obj439:head) v3 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:41ea52ba:::obj210:head local-lis/les=109/110 n=162 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:424ce90f:::obj439:head) 2026-02-11T23:31:17.208+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:424ce90f:::obj439:head) v3 finish latency 0.000012 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.208+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x497f080 con 0x3752000 2026-02-11T23:31:17.208+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:42d0e5ef:::obj468:head)]) -- 0x4955500 con 0x3752000 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.208+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 364 ==== MOSDPGPush(1.0 110/109 [PushOp(1:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39711e0 con 0x3752000 2026-02-11T23:31:17.208+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.208+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.208+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 365 ==== pg_backfill(progress 1.0 e 110/110 lb 1:42d0e5ef:::obj468:head) ==== 1062+0+0 (unknown 0 0 0) 0x4969700 con 0x3752000 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.208+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:42d0e5ef:::obj468:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.208+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:42d0e5ef:::obj468:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 110/109 [PushOp(1:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:42d0e5ef:::obj468:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:436594ec:::obj300:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:436594ec:::obj300:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:436594ec:::obj300:head 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:436594ec:::obj300:head v 107'300 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.208+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.208+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.208+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:42d0e5ef:::obj468:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:42d0e5ef:::obj468:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 110/110 lb 1:42d0e5ef:::obj468:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:42d0e5ef:::obj468:head) v3 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:424ce90f:::obj439:head local-lis/les=109/110 n=163 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:42d0e5ef:::obj468:head) 2026-02-11T23:31:17.208+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.208+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:42d0e5ef:::obj468:head) v3 finish latency 0.000012 2026-02-11T23:31:17.208+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.209+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.209+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.209+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x497f500 con 0x3752000 2026-02-11T23:31:17.209+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:436594ec:::obj300:head)]) -- 0x497f200 con 0x3752000 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.209+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 366 ==== MOSDPGPush(1.0 110/109 [PushOp(1:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3968d00 con 0x3752000 2026-02-11T23:31:17.209+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.209+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.209+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 367 ==== pg_backfill(progress 1.0 e 110/110 lb 1:436594ec:::obj300:head) ==== 1062+0+0 (unknown 0 0 0) 0x4994000 con 0x3752000 2026-02-11T23:31:17.209+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:436594ec:::obj300:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 110 2026-02-11T23:31:17.209+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.209+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:436594ec:::obj300:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.209+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 110/109 [PushOp(1:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.209+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.209+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.209+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.209+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.209+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:436594ec:::obj300:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.209+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:454d8265:::obj256:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:454d8265:::obj256:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.209+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:454d8265:::obj256:head 2026-02-11T23:31:17.209+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:454d8265:::obj256:head v 107'256 2026-02-11T23:31:17.209+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.209+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.209+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.209+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.209+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:436594ec:::obj300:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.209+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:436594ec:::obj300:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 110/110 lb 1:436594ec:::obj300:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.209+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.209+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:436594ec:::obj300:head) v3 2026-02-11T23:31:17.209+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:42d0e5ef:::obj468:head local-lis/les=109/110 n=164 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:436594ec:::obj300:head) 2026-02-11T23:31:17.209+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.209+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:436594ec:::obj300:head) v3 finish latency 0.000013 2026-02-11T23:31:17.209+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.209+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.209+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.209+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x497f980 con 0x3752000 2026-02-11T23:31:17.209+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:454d8265:::obj256:head)]) -- 0x497f680 con 0x3752000 2026-02-11T23:31:17.209+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.210+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 368 ==== MOSDPGPush(1.0 110/109 [PushOp(1:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3970820 con 0x3752000 2026-02-11T23:31:17.210+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.210+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.210+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 369 ==== pg_backfill(progress 1.0 e 110/110 lb 1:454d8265:::obj256:head) ==== 1062+0+0 (unknown 0 0 0) 0x4994580 con 0x3752000 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.210+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:454d8265:::obj256:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.210+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:454d8265:::obj256:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 110/109 [PushOp(1:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.210+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.210+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:454d8265:::obj256:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:461f8b5e:::obj16:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:461f8b5e:::obj16:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:461f8b5e:::obj16:head 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:461f8b5e:::obj16:head v 107'16 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.210+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.210+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.210+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:454d8265:::obj256:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.210+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:454d8265:::obj256:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 1.0 e 110/110 lb 1:454d8265:::obj256:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.210+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.210+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:454d8265:::obj256:head) v3 2026-02-11T23:31:17.210+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:436594ec:::obj300:head local-lis/les=109/110 n=165 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:454d8265:::obj256:head) 2026-02-11T23:31:17.210+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.210+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:454d8265:::obj256:head) v3 finish latency 0.000015 2026-02-11T23:31:17.210+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.210+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.210+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x497fc80 con 0x3752000 2026-02-11T23:31:17.210+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:461f8b5e:::obj16:head)]) -- 0x497fb00 con 0x3752000 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.210+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 370 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3970d00 con 0x3752000 2026-02-11T23:31:17.210+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.210+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.210+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 371 ==== pg_backfill(progress 1.0 e 110/110 lb 1:461f8b5e:::obj16:head) ==== 1061+0+0 (unknown 0 0 0) 0x4994b00 con 0x3752000 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.210+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:461f8b5e:::obj16:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.210+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:461f8b5e:::obj16:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.210+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.210+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:461f8b5e:::obj16:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4667f6bf:::obj584:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4667f6bf:::obj584:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4667f6bf:::obj584:head 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4667f6bf:::obj584:head v 107'584 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.210+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.210+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.210+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.210+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.210+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:461f8b5e:::obj16:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.210+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:461f8b5e:::obj16:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 110/110 lb 1:461f8b5e:::obj16:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.210+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.210+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:461f8b5e:::obj16:head) v3 2026-02-11T23:31:17.210+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:454d8265:::obj256:head local-lis/les=109/110 n=166 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:461f8b5e:::obj16:head) 2026-02-11T23:31:17.210+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.210+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:461f8b5e:::obj16:head) v3 finish latency 0.000011 2026-02-11T23:31:17.210+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.211+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.211+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.211+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a2180 con 0x3752000 2026-02-11T23:31:17.211+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4667f6bf:::obj584:head)]) -- 0x497fe00 con 0x3752000 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.211+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 372 ==== MOSDPGPush(1.0 110/109 [PushOp(1:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39701a0 con 0x3752000 2026-02-11T23:31:17.211+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.211+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.211+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 373 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4667f6bf:::obj584:head) ==== 1062+0+0 (unknown 0 0 0) 0x4995080 con 0x3752000 2026-02-11T23:31:17.211+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.211+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4667f6bf:::obj584:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.211+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4667f6bf:::obj584:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.211+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.211+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.211+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.211+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.211+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4667f6bf:::obj584:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.211+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.211+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.211+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:46b5a6d0:::obj580:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.211+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:46b5a6d0:::obj580:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.211+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:46b5a6d0:::obj580:head 2026-02-11T23:31:17.211+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:46b5a6d0:::obj580:head v 107'580 2026-02-11T23:31:17.211+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.211+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.211+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.211+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4667f6bf:::obj584:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.211+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4667f6bf:::obj584:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 110/110 lb 1:4667f6bf:::obj584:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.211+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.211+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4667f6bf:::obj584:head) v3 2026-02-11T23:31:17.211+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:461f8b5e:::obj16:head local-lis/les=109/110 n=167 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4667f6bf:::obj584:head) 2026-02-11T23:31:17.211+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.211+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4667f6bf:::obj584:head) v3 finish latency 0.000015 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.211+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.211+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.211+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a2300 con 0x3752000 2026-02-11T23:31:17.211+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:46b5a6d0:::obj580:head)]) -- 0x4955800 con 0x3752000 2026-02-11T23:31:17.211+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.212+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 374 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3970680 con 0x3752000 2026-02-11T23:31:17.212+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.212+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.212+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 375 ==== pg_backfill(progress 1.0 e 110/110 lb 1:46b5a6d0:::obj580:head) ==== 1062+0+0 (unknown 0 0 0) 0x4995600 con 0x3752000 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.212+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:46b5a6d0:::obj580:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.212+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:46b5a6d0:::obj580:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:46b5a6d0:::obj580:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 110/109 [PushOp(1:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4851a45a:::obj44:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4851a45a:::obj44:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4851a45a:::obj44:head 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4851a45a:::obj44:head v 107'44 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.212+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:46b5a6d0:::obj580:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:46b5a6d0:::obj580:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 110/110 lb 1:46b5a6d0:::obj580:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:46b5a6d0:::obj580:head) v3 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4667f6bf:::obj584:head local-lis/les=109/110 n=168 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:46b5a6d0:::obj580:head) 2026-02-11T23:31:17.212+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:46b5a6d0:::obj580:head) v3 finish latency 0.000012 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.212+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a2600 con 0x3752000 2026-02-11T23:31:17.212+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4851a45a:::obj44:head)]) -- 0x49a2480 con 0x3752000 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.212+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 376 ==== MOSDPGPush(1.0 110/109 [PushOp(1:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3969a00 con 0x3752000 2026-02-11T23:31:17.212+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.212+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.212+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 377 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4851a45a:::obj44:head) ==== 1061+0+0 (unknown 0 0 0) 0x4995b80 con 0x3752000 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.212+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4851a45a:::obj44:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.212+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4851a45a:::obj44:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4851a45a:::obj44:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:48c1458b:::obj274:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:48c1458b:::obj274:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:48c1458b:::obj274:head 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:48c1458b:::obj274:head v 107'274 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.212+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.212+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4851a45a:::obj44:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4851a45a:::obj44:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 110/110 lb 1:4851a45a:::obj44:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4851a45a:::obj44:head) v3 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:46b5a6d0:::obj580:head local-lis/les=109/110 n=169 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4851a45a:::obj44:head) 2026-02-11T23:31:17.212+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4851a45a:::obj44:head) v3 finish latency 0.000012 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.212+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.212+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.212+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a2780 con 0x3752000 2026-02-11T23:31:17.212+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:48c1458b:::obj274:head)]) -- 0x4955b00 con 0x3752000 2026-02-11T23:31:17.212+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.213+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 378 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39744e0 con 0x3752000 2026-02-11T23:31:17.213+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.213+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.213+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 379 ==== pg_backfill(progress 1.0 e 110/110 lb 1:48c1458b:::obj274:head) ==== 1062+0+0 (unknown 0 0 0) 0x4996100 con 0x3752000 2026-02-11T23:31:17.213+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.213+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:48c1458b:::obj274:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.213+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.213+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:48c1458b:::obj274:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:48c1458b:::obj274:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.213+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.213+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.213+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 110/109 [PushOp(1:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.213+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.213+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.213+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.213+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4911cfa9:::obj108:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.213+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4911cfa9:::obj108:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.213+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4911cfa9:::obj108:head 2026-02-11T23:31:17.213+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4911cfa9:::obj108:head v 107'108 2026-02-11T23:31:17.213+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.213+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.213+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.213+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:48c1458b:::obj274:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.213+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:48c1458b:::obj274:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 1.0 e 110/110 lb 1:48c1458b:::obj274:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.213+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.213+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:48c1458b:::obj274:head) v3 2026-02-11T23:31:17.213+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4851a45a:::obj44:head local-lis/les=109/110 n=170 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:48c1458b:::obj274:head) 2026-02-11T23:31:17.213+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.213+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:48c1458b:::obj274:head) v3 finish latency 0.000012 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.213+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.213+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.213+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a2900 con 0x3752000 2026-02-11T23:31:17.213+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4911cfa9:::obj108:head)]) -- 0x4955e00 con 0x3752000 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.213+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 380 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39749c0 con 0x3752000 2026-02-11T23:31:17.213+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.213+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.213+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 381 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4911cfa9:::obj108:head) ==== 1062+0+0 (unknown 0 0 0) 0x4996680 con 0x3752000 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.213+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4911cfa9:::obj108:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 110 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.213+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4911cfa9:::obj108:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.213+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.213+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.213+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 110/109 [PushOp(1:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4911cfa9:::obj108:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4918aa91:::obj303:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4918aa91:::obj303:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4918aa91:::obj303:head 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4918aa91:::obj303:head v 107'303 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.214+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4911cfa9:::obj108:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4911cfa9:::obj108:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 110/110 lb 1:4911cfa9:::obj108:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4911cfa9:::obj108:head) v3 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:48c1458b:::obj274:head local-lis/les=109/110 n=171 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4911cfa9:::obj108:head) 2026-02-11T23:31:17.214+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4911cfa9:::obj108:head) v3 finish latency 0.000011 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.214+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a2d80 con 0x3752000 2026-02-11T23:31:17.214+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4918aa91:::obj303:head)]) -- 0x49a2a80 con 0x3752000 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.214+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 382 ==== MOSDPGPush(1.0 110/109 [PushOp(1:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x39716c0 con 0x3752000 2026-02-11T23:31:17.214+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.214+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.214+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 383 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4918aa91:::obj303:head) ==== 1062+0+0 (unknown 0 0 0) 0x4996c00 con 0x3752000 2026-02-11T23:31:17.214+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4918aa91:::obj303:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.214+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4918aa91:::obj303:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 110/109 [PushOp(1:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4918aa91:::obj303:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:492a75ed:::obj72:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:492a75ed:::obj72:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:492a75ed:::obj72:head 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:492a75ed:::obj72:head v 107'72 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.214+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.214+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4918aa91:::obj303:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4918aa91:::obj303:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 110/110 lb 1:4918aa91:::obj303:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4918aa91:::obj303:head) v3 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4911cfa9:::obj108:head local-lis/les=109/110 n=172 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4918aa91:::obj303:head) 2026-02-11T23:31:17.214+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4918aa91:::obj303:head) v3 finish latency 0.000013 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.214+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.214+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.214+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a2f00 con 0x3752000 2026-02-11T23:31:17.214+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:492a75ed:::obj72:head)]) -- 0x49b8180 con 0x3752000 2026-02-11T23:31:17.214+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.215+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 384 ==== MOSDPGPush(1.0 110/109 [PushOp(1:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3971ba0 con 0x3752000 2026-02-11T23:31:17.215+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.215+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.215+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 385 ==== pg_backfill(progress 1.0 e 110/110 lb 1:492a75ed:::obj72:head) ==== 1061+0+0 (unknown 0 0 0) 0x4997180 con 0x3752000 2026-02-11T23:31:17.215+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.215+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:492a75ed:::obj72:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.215+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:492a75ed:::obj72:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.215+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.215+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.215+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.215+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.215+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.215+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.215+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:492a75ed:::obj72:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.215+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.215+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.215+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:498092df:::obj260:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.215+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:498092df:::obj260:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.215+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:498092df:::obj260:head 2026-02-11T23:31:17.215+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:498092df:::obj260:head v 107'260 2026-02-11T23:31:17.215+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.215+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.215+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.215+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.215+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.215+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:492a75ed:::obj72:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.215+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:492a75ed:::obj72:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 110/110 lb 1:492a75ed:::obj72:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.215+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.215+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:492a75ed:::obj72:head) v3 2026-02-11T23:31:17.215+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4918aa91:::obj303:head local-lis/les=109/110 n=173 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:492a75ed:::obj72:head) 2026-02-11T23:31:17.215+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.215+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:492a75ed:::obj72:head) v3 finish latency 0.000013 2026-02-11T23:31:17.215+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.215+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.215+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.215+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.215+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.215+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.215+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.215+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a3080 con 0x3752000 2026-02-11T23:31:17.215+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:498092df:::obj260:head)]) -- 0x49b8300 con 0x3752000 2026-02-11T23:31:17.215+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.215+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 386 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x449cd00 con 0x3752000 2026-02-11T23:31:17.215+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.215+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.215+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 387 ==== pg_backfill(progress 1.0 e 110/110 lb 1:498092df:::obj260:head) ==== 1062+0+0 (unknown 0 0 0) 0x4997700 con 0x3752000 2026-02-11T23:31:17.215+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.215+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:498092df:::obj260:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.215+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:498092df:::obj260:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.215+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.216+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.216+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 110/109 [PushOp(1:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:498092df:::obj260:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4987b93a:::obj56:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.216+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4987b93a:::obj56:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4987b93a:::obj56:head 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4987b93a:::obj56:head v 107'56 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.216+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.216+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:498092df:::obj260:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:498092df:::obj260:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 110/110 lb 1:498092df:::obj260:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:498092df:::obj260:head) v3 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:492a75ed:::obj72:head local-lis/les=109/110 n=174 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:498092df:::obj260:head) 2026-02-11T23:31:17.216+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:498092df:::obj260:head) v3 finish latency 0.000013 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.216+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.216+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a3200 con 0x3752000 2026-02-11T23:31:17.216+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4987b93a:::obj56:head)]) -- 0x49b8600 con 0x3752000 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.216+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 388 ==== MOSDPGPush(1.0 110/109 [PushOp(1:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x449d1e0 con 0x3752000 2026-02-11T23:31:17.216+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.216+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.216+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 389 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4987b93a:::obj56:head) ==== 1061+0+0 (unknown 0 0 0) 0x49c4000 con 0x3752000 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.216+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4987b93a:::obj56:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.216+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4987b93a:::obj56:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.216+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.216+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4987b93a:::obj56:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:498e91f7:::obj463:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:498e91f7:::obj463:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:498e91f7:::obj463:head 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:498e91f7:::obj463:head v 107'463 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.216+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.216+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.216+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4987b93a:::obj56:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4987b93a:::obj56:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 110/110 lb 1:4987b93a:::obj56:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4987b93a:::obj56:head) v3 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498092df:::obj260:head local-lis/les=109/110 n=175 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4987b93a:::obj56:head) 2026-02-11T23:31:17.216+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.216+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4987b93a:::obj56:head) v3 finish latency 0.000013 2026-02-11T23:31:17.216+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.216+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.216+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.216+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a3680 con 0x3752000 2026-02-11T23:31:17.216+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:498e91f7:::obj463:head)]) -- 0x49a3380 con 0x3752000 2026-02-11T23:31:17.216+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 390 ==== MOSDPGPush(1.0 110/109 [PushOp(1:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39656c0 con 0x3752000 2026-02-11T23:31:17.217+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.217+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 391 ==== pg_backfill(progress 1.0 e 110/110 lb 1:498e91f7:::obj463:head) ==== 1062+0+0 (unknown 0 0 0) 0x49c4580 con 0x3752000 2026-02-11T23:31:17.217+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:498e91f7:::obj463:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.217+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:498e91f7:::obj463:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.217+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.217+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.217+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:498e91f7:::obj463:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.217+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.217+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:49f89eae:::obj182:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:49f89eae:::obj182:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.217+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:49f89eae:::obj182:head 2026-02-11T23:31:17.217+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:49f89eae:::obj182:head v 107'182 2026-02-11T23:31:17.217+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.217+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.217+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:498e91f7:::obj463:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:498e91f7:::obj463:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 110/110 lb 1:498e91f7:::obj463:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:498e91f7:::obj463:head) v3 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4987b93a:::obj56:head local-lis/les=109/110 n=176 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:498e91f7:::obj463:head) 2026-02-11T23:31:17.217+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:498e91f7:::obj463:head) v3 finish latency 0.000014 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.217+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.217+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a3b00 con 0x3752000 2026-02-11T23:31:17.217+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:49f89eae:::obj182:head)]) -- 0x49a3800 con 0x3752000 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 392 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x449c820 con 0x3752000 2026-02-11T23:31:17.217+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.217+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.217+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 393 ==== pg_backfill(progress 1.0 e 110/110 lb 1:49f89eae:::obj182:head) ==== 1062+0+0 (unknown 0 0 0) 0x49c4b00 con 0x3752000 2026-02-11T23:31:17.217+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:49f89eae:::obj182:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.217+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:49f89eae:::obj182:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 110/109 [PushOp(1:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.217+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:49f89eae:::obj182:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4b1d3252:::obj86:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.217+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4b1d3252:::obj86:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4b1d3252:::obj86:head 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4b1d3252:::obj86:head v 107'86 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.217+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.217+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:49f89eae:::obj182:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.218+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:49f89eae:::obj182:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 110/110 lb 1:49f89eae:::obj182:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:49f89eae:::obj182:head) v3 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:498e91f7:::obj463:head local-lis/les=109/110 n=177 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:49f89eae:::obj182:head) 2026-02-11T23:31:17.218+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:49f89eae:::obj182:head) v3 finish latency 0.000017 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.218+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.218+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49a3c80 con 0x3752000 2026-02-11T23:31:17.218+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4b1d3252:::obj86:head)]) -- 0x49b8780 con 0x3752000 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.218+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 394 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x268a1a0 con 0x3752000 2026-02-11T23:31:17.218+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.218+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.218+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 395 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4b1d3252:::obj86:head) ==== 1061+0+0 (unknown 0 0 0) 0x49c5080 con 0x3752000 2026-02-11T23:31:17.218+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4b1d3252:::obj86:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.218+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4b1d3252:::obj86:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 110/109 [PushOp(1:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.218+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.218+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4b1d3252:::obj86:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4b57ab44:::obj335:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4b57ab44:::obj335:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4b57ab44:::obj335:head 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4b57ab44:::obj335:head v 107'335 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.218+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.218+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.218+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4b1d3252:::obj86:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.218+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4b1d3252:::obj86:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 110/110 lb 1:4b1d3252:::obj86:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.218+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.218+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4b1d3252:::obj86:head) v3 2026-02-11T23:31:17.218+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:49f89eae:::obj182:head local-lis/les=109/110 n=178 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4b1d3252:::obj86:head) 2026-02-11T23:31:17.218+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.218+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4b1d3252:::obj86:head) v3 finish latency 0.000013 2026-02-11T23:31:17.218+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.218+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.218+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.218+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49d6180 con 0x3752000 2026-02-11T23:31:17.218+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4b57ab44:::obj335:head)]) -- 0x49a3e00 con 0x3752000 2026-02-11T23:31:17.218+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.219+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 396 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3965520 con 0x3752000 2026-02-11T23:31:17.219+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.219+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.219+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 397 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4b57ab44:::obj335:head) ==== 1062+0+0 (unknown 0 0 0) 0x49c5600 con 0x3752000 2026-02-11T23:31:17.219+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4b57ab44:::obj335:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.219+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4b57ab44:::obj335:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 110/109 [PushOp(1:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4b57ab44:::obj335:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4b639beb:::obj280:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4b639beb:::obj280:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4b639beb:::obj280:head 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4b639beb:::obj280:head v 107'280 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.219+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4b57ab44:::obj335:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.219+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4b57ab44:::obj335:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 110/110 lb 1:4b57ab44:::obj335:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.219+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.219+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4b57ab44:::obj335:head) v3 2026-02-11T23:31:17.219+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b1d3252:::obj86:head local-lis/les=109/110 n=179 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4b57ab44:::obj335:head) 2026-02-11T23:31:17.219+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.219+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4b57ab44:::obj335:head) v3 finish latency 0.000013 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.219+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49d6600 con 0x3752000 2026-02-11T23:31:17.219+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4b639beb:::obj280:head)]) -- 0x49d6300 con 0x3752000 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.219+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 398 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3964b60 con 0x3752000 2026-02-11T23:31:17.219+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 110 2026-02-11T23:31:17.219+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.219+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 399 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4b639beb:::obj280:head) ==== 1062+0+0 (unknown 0 0 0) 0x49c5b80 con 0x3752000 2026-02-11T23:31:17.219+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4b639beb:::obj280:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 110 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.219+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4b639beb:::obj280:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4b639beb:::obj280:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4baa8bb7:::obj594:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4baa8bb7:::obj594:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4baa8bb7:::obj594:head 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4baa8bb7:::obj594:head v 107'594 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.219+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.219+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.219+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4b639beb:::obj280:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.219+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4b639beb:::obj280:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 110/110 lb 1:4b639beb:::obj280:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.219+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.219+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4b639beb:::obj280:head) v3 2026-02-11T23:31:17.219+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b57ab44:::obj335:head local-lis/les=109/110 n=180 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4b639beb:::obj280:head) 2026-02-11T23:31:17.219+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.219+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4b639beb:::obj280:head) v3 finish latency 0.000017 2026-02-11T23:31:17.219+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.220+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.220+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.220+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49d6a80 con 0x3752000 2026-02-11T23:31:17.220+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4baa8bb7:::obj594:head)]) -- 0x49d6780 con 0x3752000 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.220+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.220+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 400 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3965040 con 0x3752000 2026-02-11T23:31:17.220+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.220+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.220+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.220+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 110/109 [PushOp(1:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.220+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.220+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.220+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 401 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4baa8bb7:::obj594:head) ==== 1062+0+0 (unknown 0 0 0) 0x49c6100 con 0x3752000 2026-02-11T23:31:17.220+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.220+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4baa8bb7:::obj594:head) v3 prio 15 type 95 cost 0 latency 0.000013 epoch 110 2026-02-11T23:31:17.220+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4c3f91c8:::obj470:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4c3f91c8:::obj470:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.220+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4baa8bb7:::obj594:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.220+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4c3f91c8:::obj470:head 2026-02-11T23:31:17.220+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4c3f91c8:::obj470:head v 107'470 2026-02-11T23:31:17.220+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.220+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.220+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.220+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4baa8bb7:::obj594:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.220+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.220+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.220+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4baa8bb7:::obj594:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.220+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4baa8bb7:::obj594:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 110/110 lb 1:4baa8bb7:::obj594:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.220+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.220+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4baa8bb7:::obj594:head) v3 2026-02-11T23:31:17.220+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4b639beb:::obj280:head local-lis/les=109/110 n=181 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4baa8bb7:::obj594:head) 2026-02-11T23:31:17.220+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.220+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4baa8bb7:::obj594:head) v3 finish latency 0.000027 2026-02-11T23:31:17.220+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.220+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.220+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.220+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49d6f00 con 0x3752000 2026-02-11T23:31:17.220+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4c3f91c8:::obj470:head)]) -- 0x49d6c00 con 0x3752000 2026-02-11T23:31:17.220+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.221+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 402 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3a00 con 0x3752000 2026-02-11T23:31:17.221+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000021 epoch 110 2026-02-11T23:31:17.221+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.221+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.221+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 403 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4c3f91c8:::obj470:head) ==== 1062+0+0 (unknown 0 0 0) 0x49c6680 con 0x3752000 2026-02-11T23:31:17.221+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.221+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c3f91c8:::obj470:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 110 2026-02-11T23:31:17.221+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.221+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c3f91c8:::obj470:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.221+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.221+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.221+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000058 MOSDPGPush(1.0 110/109 [PushOp(1:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.221+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.221+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.221+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.221+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c3f91c8:::obj470:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.221+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.221+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4c59dc70:::obj344:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.221+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4c59dc70:::obj344:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.221+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4c59dc70:::obj344:head 2026-02-11T23:31:17.221+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4c59dc70:::obj344:head v 107'344 2026-02-11T23:31:17.221+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.221+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.221+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:17.221+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.221+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.221+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c3f91c8:::obj470:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.221+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c3f91c8:::obj470:head) v3 prio 15 cost 0 latency 0.000101 pg_backfill(progress 1.0 e 110/110 lb 1:4c3f91c8:::obj470:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.221+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.221+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4c3f91c8:::obj470:head) v3 2026-02-11T23:31:17.221+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4baa8bb7:::obj594:head local-lis/les=109/110 n=182 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4c3f91c8:::obj470:head) 2026-02-11T23:31:17.221+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.221+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c3f91c8:::obj470:head) v3 finish latency 0.000015 2026-02-11T23:31:17.221+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.221+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.221+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.221+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.221+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.221+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.221+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.221+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49d7080 con 0x3752000 2026-02-11T23:31:17.221+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4c59dc70:::obj344:head)]) -- 0x49b8900 con 0x3752000 2026-02-11T23:31:17.221+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.221+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 404 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x268ad00 con 0x3752000 2026-02-11T23:31:17.221+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000019 epoch 110 2026-02-11T23:31:17.222+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.222+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 405 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4c59dc70:::obj344:head) ==== 1062+0+0 (unknown 0 0 0) 0x49c6c00 con 0x3752000 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.222+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c59dc70:::obj344:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 110 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.222+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c59dc70:::obj344:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000057 MOSDPGPush(1.0 110/109 [PushOp(1:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c59dc70:::obj344:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4c73ba95:::obj68:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4c73ba95:::obj68:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4c73ba95:::obj68:head 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4c73ba95:::obj68:head v 107'68 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.222+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c59dc70:::obj344:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.222+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c59dc70:::obj344:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 110/110 lb 1:4c59dc70:::obj344:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.222+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.222+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4c59dc70:::obj344:head) v3 2026-02-11T23:31:17.222+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c3f91c8:::obj470:head local-lis/les=109/110 n=183 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4c59dc70:::obj344:head) 2026-02-11T23:31:17.222+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.222+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c59dc70:::obj344:head) v3 finish latency 0.000023 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.222+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49d7380 con 0x3752000 2026-02-11T23:31:17.222+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4c73ba95:::obj68:head)]) -- 0x49d7200 con 0x3752000 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.222+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 406 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3965d40 con 0x3752000 2026-02-11T23:31:17.222+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 110 2026-02-11T23:31:17.222+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.222+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 407 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4c73ba95:::obj68:head) ==== 1061+0+0 (unknown 0 0 0) 0x49c7180 con 0x3752000 2026-02-11T23:31:17.222+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c73ba95:::obj68:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.222+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c73ba95:::obj68:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 110/109 [PushOp(1:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c73ba95:::obj68:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4c7e2cfa:::obj454:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4c7e2cfa:::obj454:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4c7e2cfa:::obj454:head 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4c7e2cfa:::obj454:head v 107'454 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.222+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c73ba95:::obj68:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.222+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c73ba95:::obj68:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 110/110 lb 1:4c73ba95:::obj68:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.222+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.222+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4c73ba95:::obj68:head) v3 2026-02-11T23:31:17.222+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c59dc70:::obj344:head local-lis/les=109/110 n=184 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4c73ba95:::obj68:head) 2026-02-11T23:31:17.222+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.222+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c73ba95:::obj68:head) v3 finish latency 0.000017 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.222+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.222+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.222+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49d7800 con 0x3752000 2026-02-11T23:31:17.222+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4c7e2cfa:::obj454:head)]) -- 0x49d7500 con 0x3752000 2026-02-11T23:31:17.222+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.223+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 408 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3040 con 0x3752000 2026-02-11T23:31:17.223+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000018 epoch 110 2026-02-11T23:31:17.223+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.223+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.223+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 409 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4c7e2cfa:::obj454:head) ==== 1062+0+0 (unknown 0 0 0) 0x49c7700 con 0x3752000 2026-02-11T23:31:17.223+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c7e2cfa:::obj454:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 110 2026-02-11T23:31:17.223+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.223+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c7e2cfa:::obj454:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.223+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.223+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.223+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000052 MOSDPGPush(1.0 110/109 [PushOp(1:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.223+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.223+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.223+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c7e2cfa:::obj454:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.223+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.223+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.223+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4c9f7fa8:::obj477:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.223+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4c9f7fa8:::obj477:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.223+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4c9f7fa8:::obj477:head 2026-02-11T23:31:17.223+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4c9f7fa8:::obj477:head v 107'477 2026-02-11T23:31:17.223+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.223+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.223+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.223+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.223+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.223+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c7e2cfa:::obj454:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.223+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c7e2cfa:::obj454:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 1.0 e 110/110 lb 1:4c7e2cfa:::obj454:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.223+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.223+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4c7e2cfa:::obj454:head) v3 2026-02-11T23:31:17.223+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c73ba95:::obj68:head local-lis/les=109/110 n=185 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4c7e2cfa:::obj454:head) 2026-02-11T23:31:17.223+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.223+0000 7fac55170640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c7e2cfa:::obj454:head) v3 finish latency 0.000015 2026-02-11T23:31:17.223+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.223+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.223+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.223+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.223+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.223+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.223+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.223+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49d7980 con 0x3752000 2026-02-11T23:31:17.223+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4c9f7fa8:::obj477:head)]) -- 0x49b8c00 con 0x3752000 2026-02-11T23:31:17.223+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.224+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 410 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3d40 con 0x3752000 2026-02-11T23:31:17.224+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 110 2026-02-11T23:31:17.224+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 411 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4c9f7fa8:::obj477:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3752000 2026-02-11T23:31:17.224+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c9f7fa8:::obj477:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 110 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.224+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c9f7fa8:::obj477:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 110/109 [PushOp(1:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c9f7fa8:::obj477:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4dc3027d:::obj550:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4dc3027d:::obj550:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4dc3027d:::obj550:head 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4dc3027d:::obj550:head v 107'550 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.224+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4c9f7fa8:::obj477:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c9f7fa8:::obj477:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 110/110 lb 1:4c9f7fa8:::obj477:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4c9f7fa8:::obj477:head) v3 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c7e2cfa:::obj454:head local-lis/les=109/110 n=186 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4c9f7fa8:::obj477:head) 2026-02-11T23:31:17.224+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 110 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4c9f7fa8:::obj477:head) v3 finish latency 0.000023 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 110 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 110 get_nextmap_reserved map_reservations: {110=1} 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 110 release_map epoch: 110 2026-02-11T23:31:17.224+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x49d7e00 con 0x3752000 2026-02-11T23:31:17.224+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4dc3027d:::obj550:head)]) -- 0x49d7b00 con 0x3752000 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.224+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 38 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 111) e111/111) ==== 158+0+0 (unknown 0 0 0) 0x46c85a0 con 0x3e4a000 2026-02-11T23:31:17.224+0000 7fac703fa640 15 osd.0 110 enqueue_peering_evt 4.0 epoch_sent: 111 epoch_requested: 111 MQuery 4.0 from 4 query_epoch 111 query: query(info 0'0 epoch_sent 111) 2026-02-11T23:31:17.224+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 MQuery 4.0 from 4 query_epoch 111 query: query(info 0'0 epoch_sent 111)) class_id 2 prio 255 cost 10 e111) 2026-02-11T23:31:17.224+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 37 ==== osd_map(108..111 src has 1..111) ==== 2075+0+0 (unknown 0 0 0) 0x46ea380 con 0x3e4a000 2026-02-11T23:31:17.224+0000 7fac606e4640 20 osd.0 110 OSD::ms_dispatch: osd_map(108..111 src has 1..111) 2026-02-11T23:31:17.224+0000 7fac606e4640 20 osd.0 110 _dispatch 0x46ea380 osd_map(108..111 src has 1..111) 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 MQuery 4.0 from 4 query_epoch 111 query: query(info 0'0 epoch_sent 111)) class_id 2immediate prio 255 cost 10 e111) queued 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 MQuery 4.0 from 4 query_epoch 111 query: query(info 0'0 epoch_sent 111)) class_id 2immediate prio 255 cost 10 e111) pg 0 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 map 111 > 110 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 111, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 MQuery 4.0 from 4 query_epoch 111 query: query(info 0'0 epoch_sent 111)) class_id 2immediate prio 255 cost 10 e111) 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.224+0000 7fac606e4640 3 osd.0 110 handle_osd_map epochs [108,111], i have 110, src has [1,111] 2026-02-11T23:31:17.224+0000 7fac606e4640 10 osd.0 110 handle_osd_map: received 0 full maps and 4 incremental maps 2026-02-11T23:31:17.224+0000 7fac606e4640 10 osd.0 110 handle_osd_map got inc map for epoch 111 2026-02-11T23:31:17.224+0000 7fac606e4640 20 osd.0 110 got_full_map 111, nothing requested 2026-02-11T23:31:17.224+0000 7fac606e4640 20 osd.0 110 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:17.224+0000 7fac606e4640 20 osd.0 110 handle_osd_map pg_num_history pg_num_history(e111 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:31:17.224+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {111={}} 2026-02-11T23:31:17.224+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:17.224+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e111 maps [1~111] lci=[5,111] tlb=1) 2026-02-11T23:31:17.224+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 412 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x44ae680 con 0x3752000 2026-02-11T23:31:17.224+0000 7fac703fa640 15 osd.0 110 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 110 2026-02-11T23:31:17.224+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 413 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4dc3027d:::obj550:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4580 con 0x3752000 2026-02-11T23:31:17.224+0000 7fac703fa640 15 osd.0 110 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4dc3027d:::obj550:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 110 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.224+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4dc3027d:::obj550:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.224+0000 7fac5e983640 10 osd.0 110 _committed_osd_maps 111..111 2026-02-11T23:31:17.224+0000 7fac5e983640 10 osd.0 110 advance to epoch 111 (<= last 111 <= newest_map 111) 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 110 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 110/109 [PushOp(1:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 110 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0 111 consume_map version 111 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4dc3027d:::obj550:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0 110 await_reserved_maps epoch:111 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0 110 await_reserved_maps done 0 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0 111 consume_map 111 -- publish done 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x49cc160 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 4.0 has no pg and waiting_for_split 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x25809a0 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0 111 identify_splits_and_merges 1.0 e110 to e111 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0 111 identify_splits_and_merges 1.0 e110 to e111 pg_nums {24=1} 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0 111 prune_pg_created 2026-02-11T23:31:17.224+0000 7fac5e983640 10 osd.0:0.consume_map 111 (was 110) 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 111 <= 111, requeueing 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {111=} 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:17.224+0000 7fac5e983640 10 osd.0:1.consume_map 111 (was 110) 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4dd77094:::obj38:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.224+0000 7fac5e983640 10 osd.0:2.consume_map 111 (was 110) 2026-02-11T23:31:17.224+0000 7fac5e983640 10 osd.0:3.consume_map 111 (was 110) 2026-02-11T23:31:17.224+0000 7fac5e983640 10 osd.0:4.consume_map 111 (was 110) 2026-02-11T23:31:17.224+0000 7fac5e983640 10 osd.0:5.consume_map 111 (was 110) 2026-02-11T23:31:17.224+0000 7fac5e983640 10 osd.0:6.consume_map 111 (was 110) 2026-02-11T23:31:17.224+0000 7fac5e983640 10 osd.0:7.consume_map 111 (was 110) 2026-02-11T23:31:17.224+0000 7fac5e983640 10 osd.0 111 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4dd77094:::obj38:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.224+0000 7fac5e983640 15 osd.0 111 enqueue_peering_evt 1.0 epoch_sent: 111 epoch_requested: 111 NullEvt 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4dd77094:::obj38:head 2026-02-11T23:31:17.224+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 NullEvt) class_id 2 prio 255 cost 10 e111) 2026-02-11T23:31:17.224+0000 7fac5e983640 7 osd.0 111 activate_map version 111 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4dd77094:::obj38:head v 107'38 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.224+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.224+0000 7fac55971640 20 osd.0 111 agent_entry empty queue 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000077 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 MQuery 4.0 from 4 query_epoch 111 query: query(info 0'0 epoch_sent 111)) class_id 2immediate prio 255 cost 10 e111) queued 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 MQuery 4.0 from 4 query_epoch 111 query: query(info 0'0 epoch_sent 111)) class_id 2immediate prio 255 cost 10 e111) pg 0 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 111 handle_pg_query_nopg 4.0 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 111 pg 4.0 dne 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 111 get_con_osd_cluster to osd.4 from_epoch 111 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 107 -> 111 (shared) 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 111 send_incremental_map epoch range: (108, 111] to v2:127.0.0.1:6836/864727417 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 111 add_map_inc_bl 111 574 bytes 2026-02-11T23:31:17.224+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(108..111 src has 1..111) -- 0x39416c0 con 0x3e4a000 2026-02-11T23:31:17.224+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:111 sent:111 4.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e111/111) -- 0x49fc000 con 0x3e4a000 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4dc3027d:::obj550:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4dc3027d:::obj550:head) v3 prio 15 cost 0 latency 0.000180 pg_backfill(progress 1.0 e 110/110 lb 1:4dc3027d:::obj550:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 NullEvt) class_id 2immediate prio 255 cost 10 e111) queued 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 110 is not newer than session's projected_epoch of 110 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4dc3027d:::obj550:head) v3 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4c9f7fa8:::obj477:head local-lis/les=109/110 n=187 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4dc3027d:::obj550:head) 2026-02-11T23:31:17.224+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4dc3027d:::obj550:head) v3 finish latency 0.000019 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.224+0000 7fac55170640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 110 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.224+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e110/110) -- 0x4a04000 con 0x3752000 2026-02-11T23:31:17.224+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4dd77094:::obj38:head)]) -- 0x49b8f00 con 0x3752000 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 111 epoch_requested: 111 NullEvt) class_id 2immediate prio 255 cost 10 e111) pg 0x4a2a000 2026-02-11T23:31:17.224+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_advance_map: 111 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 110 -> 111 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 110 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1] -- 1/1 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] state: Started advmap 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 111 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_activate_map: epoch range: 111..111 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 110 while current is 111 2026-02-11T23:31:17.224+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.224+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 111 epoch_requested: 111 NullEvt 2026-02-11T23:31:17.224+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.224+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.225+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.225+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 414 ==== MOSDPGPush(1.0 110/109 [PushOp(1:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3964820 con 0x3752000 2026-02-11T23:31:17.225+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 110/109 [PushOp(1:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 110 2026-02-11T23:31:17.225+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e110) 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.225+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 415 ==== pg_backfill(progress 1.0 e 110/110 lb 1:4dd77094:::obj38:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f4b00 con 0x3752000 2026-02-11T23:31:17.225+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 110/110 lb 1:4dd77094:::obj38:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 110 2026-02-11T23:31:17.225+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4dd77094:::obj38:head) v3) class_id 0 prio 15 cost 0 e110) 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) queued 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 110/109 [PushOp(1:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e110) pg 0x4a2a000 2026-02-11T23:31:17.225+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 110/109 [PushOp(1:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.225+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 110 -> 111 (shared) 2026-02-11T23:31:17.225+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.225+0000 7fac55170640 10 osd.0 111 send_incremental_map epoch range: (111, 111] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:31:17.225+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4dd77094:::obj38:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) queued 2026-02-11T23:31:17.225+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(111..111 src has 1..111) -- 0x46eb340 con 0x3752000 2026-02-11T23:31:17.225+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 110/109 [PushOp(1:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.225+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.225+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4eb30133:::obj30:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4eb30133:::obj30:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.225+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4eb30133:::obj30:head 2026-02-11T23:31:17.225+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4eb30133:::obj30:head v 107'30 2026-02-11T23:31:17.225+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.225+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.225+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 110/109 [PushOp(1:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000146 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.225+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.225+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 110/110 lb 1:4dd77094:::obj38:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e110) pg 0x4a2a000 2026-02-11T23:31:17.225+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4dd77094:::obj38:head) v3 prio 15 cost 0 latency 0.000171 pg_backfill(progress 1.0 e 110/110 lb 1:4dd77094:::obj38:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.225+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.225+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 110/110 lb 1:4dd77094:::obj38:head) v3 2026-02-11T23:31:17.225+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dc3027d:::obj550:head local-lis/les=109/110 n=188 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 110/110 lb 1:4dd77094:::obj38:head) 2026-02-11T23:31:17.225+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.225+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 110/110 lb 1:4dd77094:::obj38:head) v3 finish latency 0.000021 2026-02-11T23:31:17.225+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.225+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.225+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.225+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4a04300 con 0x3752000 2026-02-11T23:31:17.225+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 110/109 [PushReplyOp(1:4eb30133:::obj30:head)]) -- 0x4a04180 con 0x3752000 2026-02-11T23:31:17.225+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.226+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 416 ==== MOSDPGPush(1.0 111/109 [PushOp(1:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44aeea0 con 0x3752000 2026-02-11T23:31:17.226+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.226+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 111/109 [PushOp(1:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4fbee9e6:::obj123:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4fbee9e6:::obj123:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.226+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 417 ==== osd_map(111..111 src has 1..111) ==== 618+0+0 (unknown 0 0 0) 0x46eb340 con 0x3752000 2026-02-11T23:31:17.226+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 418 ==== pg_backfill(progress 1.0 e 111/111 lb 1:4eb30133:::obj30:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5080 con 0x3752000 2026-02-11T23:31:17.226+0000 7fac606e4640 20 osd.0 111 OSD::ms_dispatch: osd_map(111..111 src has 1..111) 2026-02-11T23:31:17.226+0000 7fac606e4640 20 osd.0 111 _dispatch 0x46eb340 osd_map(111..111 src has 1..111) 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4fbee9e6:::obj123:head 2026-02-11T23:31:17.226+0000 7fac606e4640 3 osd.0 111 handle_osd_map epochs [111,111], i have 111, src has [1,111] 2026-02-11T23:31:17.226+0000 7fac606e4640 10 osd.0 111 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.226+0000 7fac606e4640 10 osd.0 111 no new maps here, dropping 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4fbee9e6:::obj123:head v 107'123 2026-02-11T23:31:17.226+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:4eb30133:::obj30:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 111 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.226+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:4eb30133:::obj30:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.226+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:4eb30133:::obj30:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000075 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:4eb30133:::obj30:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.226+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:4eb30133:::obj30:head) v3 prio 15 cost 0 latency 0.000055 pg_backfill(progress 1.0 e 111/111 lb 1:4eb30133:::obj30:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.226+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.226+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:4eb30133:::obj30:head) v3 2026-02-11T23:31:17.226+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4dd77094:::obj38:head local-lis/les=109/110 n=189 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:4eb30133:::obj30:head) 2026-02-11T23:31:17.226+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.226+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:4eb30133:::obj30:head) v3 finish latency 0.000017 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.226+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.226+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4a04480 con 0x3752000 2026-02-11T23:31:17.226+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:4fbee9e6:::obj123:head)]) -- 0x49b9080 con 0x3752000 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.226+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 419 ==== MOSDPGPush(1.0 111/109 [PushOp(1:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44ae000 con 0x3752000 2026-02-11T23:31:17.226+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 111 2026-02-11T23:31:17.226+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.226+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 420 ==== pg_backfill(progress 1.0 e 111/111 lb 1:4fbee9e6:::obj123:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5600 con 0x3752000 2026-02-11T23:31:17.226+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:4fbee9e6:::obj123:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 111 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.226+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:4fbee9e6:::obj123:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000050 MOSDPGPush(1.0 111/109 [PushOp(1:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:4fbee9e6:::obj123:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:4feb0eaf:::obj556:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:4feb0eaf:::obj556:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:4feb0eaf:::obj556:head 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:4feb0eaf:::obj556:head v 107'556 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.226+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.226+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:17.226+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:4fbee9e6:::obj123:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.226+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:4fbee9e6:::obj123:head) v3 prio 15 cost 0 latency 0.000102 pg_backfill(progress 1.0 e 111/111 lb 1:4fbee9e6:::obj123:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.226+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.226+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:4fbee9e6:::obj123:head) v3 2026-02-11T23:31:17.226+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4eb30133:::obj30:head local-lis/les=109/110 n=190 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:4fbee9e6:::obj123:head) 2026-02-11T23:31:17.226+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.226+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:4fbee9e6:::obj123:head) v3 finish latency 0.000012 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.226+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.227+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.227+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.227+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4a04600 con 0x3752000 2026-02-11T23:31:17.227+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:4feb0eaf:::obj556:head)]) -- 0x49b9380 con 0x3752000 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.227+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 421 ==== MOSDPGPush(1.0 111/109 [PushOp(1:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44ae4e0 con 0x3752000 2026-02-11T23:31:17.227+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.227+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.227+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 422 ==== pg_backfill(progress 1.0 e 111/111 lb 1:4feb0eaf:::obj556:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3752000 2026-02-11T23:31:17.227+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:4feb0eaf:::obj556:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 111 2026-02-11T23:31:17.227+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:4feb0eaf:::obj556:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.227+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.227+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.227+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.227+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.227+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 111/109 [PushOp(1:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.227+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:4feb0eaf:::obj556:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.227+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.227+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.227+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:50174016:::obj321:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.227+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:50174016:::obj321:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.227+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:50174016:::obj321:head 2026-02-11T23:31:17.227+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:50174016:::obj321:head v 107'321 2026-02-11T23:31:17.227+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.227+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.227+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:17.227+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:4feb0eaf:::obj556:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.227+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:4feb0eaf:::obj556:head) v3 prio 15 cost 0 latency 0.000104 pg_backfill(progress 1.0 e 111/111 lb 1:4feb0eaf:::obj556:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.227+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.227+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:4feb0eaf:::obj556:head) v3 2026-02-11T23:31:17.227+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4fbee9e6:::obj123:head local-lis/les=109/110 n=191 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:4feb0eaf:::obj556:head) 2026-02-11T23:31:17.227+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.227+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:4feb0eaf:::obj556:head) v3 finish latency 0.000015 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.227+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.227+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.227+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4a04780 con 0x3752000 2026-02-11T23:31:17.227+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:50174016:::obj321:head)]) -- 0x49b9680 con 0x3752000 2026-02-11T23:31:17.227+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.228+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 423 ==== MOSDPGPush(1.0 111/109 [PushOp(1:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4445040 con 0x3752000 2026-02-11T23:31:17.228+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 111 2026-02-11T23:31:17.228+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.228+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 424 ==== pg_backfill(progress 1.0 e 111/111 lb 1:50174016:::obj321:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3752000 2026-02-11T23:31:17.228+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:50174016:::obj321:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.228+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:50174016:::obj321:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 111/109 [PushOp(1:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:50174016:::obj321:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:502bc4ef:::obj531:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:502bc4ef:::obj531:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:502bc4ef:::obj531:head 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:502bc4ef:::obj531:head v 107'531 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.228+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:50174016:::obj321:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.228+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:50174016:::obj321:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 111/111 lb 1:50174016:::obj321:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.228+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.228+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:50174016:::obj321:head) v3 2026-02-11T23:31:17.228+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:4feb0eaf:::obj556:head local-lis/les=109/110 n=192 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:50174016:::obj321:head) 2026-02-11T23:31:17.228+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.228+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:50174016:::obj321:head) v3 finish latency 0.000022 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.228+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4a04c00 con 0x3752000 2026-02-11T23:31:17.228+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:502bc4ef:::obj531:head)]) -- 0x4a04900 con 0x3752000 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.228+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 425 ==== MOSDPGPush(1.0 111/109 [PushOp(1:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44444e0 con 0x3752000 2026-02-11T23:31:17.228+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 111 2026-02-11T23:31:17.228+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.228+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 426 ==== pg_backfill(progress 1.0 e 111/111 lb 1:502bc4ef:::obj531:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3752000 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.228+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:502bc4ef:::obj531:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 111 2026-02-11T23:31:17.228+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:502bc4ef:::obj531:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 111/109 [PushOp(1:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:502bc4ef:::obj531:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:506f5ba0:::obj253:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:506f5ba0:::obj253:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:506f5ba0:::obj253:head 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:506f5ba0:::obj253:head v 107'253 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.228+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.228+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:502bc4ef:::obj531:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.228+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:502bc4ef:::obj531:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 111/111 lb 1:502bc4ef:::obj531:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.228+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.228+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:502bc4ef:::obj531:head) v3 2026-02-11T23:31:17.228+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50174016:::obj321:head local-lis/les=109/110 n=193 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:502bc4ef:::obj531:head) 2026-02-11T23:31:17.228+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.228+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:502bc4ef:::obj531:head) v3 finish latency 0.000018 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.228+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.228+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.229+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4a05080 con 0x3752000 2026-02-11T23:31:17.229+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:506f5ba0:::obj253:head)]) -- 0x4a04d80 con 0x3752000 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.229+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.229+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 427 ==== MOSDPGPush(1.0 111/109 [PushOp(1:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x448e000 con 0x3752000 2026-02-11T23:31:17.229+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000026 epoch 111 2026-02-11T23:31:17.229+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.229+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 428 ==== pg_backfill(progress 1.0 e 111/111 lb 1:506f5ba0:::obj253:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3752000 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.229+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:506f5ba0:::obj253:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 111 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.229+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:506f5ba0:::obj253:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.229+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000058 MOSDPGPush(1.0 111/109 [PushOp(1:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.229+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.229+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.229+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.229+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.229+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:50fb58bc:::obj174:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:50fb58bc:::obj174:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.229+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.229+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:50fb58bc:::obj174:head 2026-02-11T23:31:17.229+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:506f5ba0:::obj253:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.229+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:50fb58bc:::obj174:head v 107'174 2026-02-11T23:31:17.229+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.229+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.229+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.229+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.229+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:506f5ba0:::obj253:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.229+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:506f5ba0:::obj253:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 111/111 lb 1:506f5ba0:::obj253:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.229+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.229+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:506f5ba0:::obj253:head) v3 2026-02-11T23:31:17.229+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:502bc4ef:::obj531:head local-lis/les=109/110 n=194 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:506f5ba0:::obj253:head) 2026-02-11T23:31:17.229+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.229+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:506f5ba0:::obj253:head) v3 finish latency 0.000027 2026-02-11T23:31:17.229+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.229+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.229+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.229+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4a05500 con 0x3752000 2026-02-11T23:31:17.229+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:50fb58bc:::obj174:head)]) -- 0x4a05200 con 0x3752000 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.229+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.229+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.230+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 429 ==== MOSDPGPush(1.0 111/109 [PushOp(1:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44ae9c0 con 0x3752000 2026-02-11T23:31:17.230+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 111 2026-02-11T23:31:17.230+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.230+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 430 ==== pg_backfill(progress 1.0 e 111/111 lb 1:50fb58bc:::obj174:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3752000 2026-02-11T23:31:17.230+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:50fb58bc:::obj174:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 111 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.230+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:50fb58bc:::obj174:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 111/109 [PushOp(1:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:50fb58bc:::obj174:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:51058f37:::obj394:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:51058f37:::obj394:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:51058f37:::obj394:head 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:51058f37:::obj394:head v 107'394 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.230+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:50fb58bc:::obj174:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:50fb58bc:::obj174:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 111/111 lb 1:50fb58bc:::obj174:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:50fb58bc:::obj174:head) v3 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:506f5ba0:::obj253:head local-lis/les=109/110 n=195 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:50fb58bc:::obj174:head) 2026-02-11T23:31:17.230+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:50fb58bc:::obj174:head) v3 finish latency 0.000019 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.230+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4a05980 con 0x3752000 2026-02-11T23:31:17.230+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:51058f37:::obj394:head)]) -- 0x4a05680 con 0x3752000 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.230+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 431 ==== MOSDPGPush(1.0 111/109 [PushOp(1:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43e4d00 con 0x3752000 2026-02-11T23:31:17.230+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 111 2026-02-11T23:31:17.230+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.230+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 432 ==== pg_backfill(progress 1.0 e 111/111 lb 1:51058f37:::obj394:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7700 con 0x3752000 2026-02-11T23:31:17.230+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:51058f37:::obj394:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 111 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.230+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:51058f37:::obj394:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 111/109 [PushOp(1:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:51058f37:::obj394:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:5170da72:::obj410:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:5170da72:::obj410:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:5170da72:::obj410:head 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:5170da72:::obj410:head v 107'410 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.230+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.230+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:51058f37:::obj394:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:51058f37:::obj394:head) v3 prio 15 cost 0 latency 0.000100 pg_backfill(progress 1.0 e 111/111 lb 1:51058f37:::obj394:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:51058f37:::obj394:head) v3 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:50fb58bc:::obj174:head local-lis/les=109/110 n=196 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:51058f37:::obj394:head) 2026-02-11T23:31:17.230+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:51058f37:::obj394:head) v3 finish latency 0.000015 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.230+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.230+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.230+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.231+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4a05b00 con 0x3752000 2026-02-11T23:31:17.231+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:5170da72:::obj410:head)]) -- 0x49b9980 con 0x3752000 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.231+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 433 ==== MOSDPGPush(1.0 111/109 [PushOp(1:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43e51e0 con 0x3752000 2026-02-11T23:31:17.231+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.231+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.231+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 434 ==== pg_backfill(progress 1.0 e 111/111 lb 1:5170da72:::obj410:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f28000 con 0x3752000 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.231+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:5170da72:::obj410:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.231+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5170da72:::obj410:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.231+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 111/109 [PushOp(1:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.231+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.231+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.231+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5170da72:::obj410:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.231+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.231+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.231+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:5178a392:::obj281:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:5178a392:::obj281:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.231+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:5178a392:::obj281:head 2026-02-11T23:31:17.231+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:5178a392:::obj281:head v 107'281 2026-02-11T23:31:17.231+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.231+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.231+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.231+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.231+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5170da72:::obj410:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.231+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5170da72:::obj410:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 111/111 lb 1:5170da72:::obj410:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.231+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.231+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:5170da72:::obj410:head) v3 2026-02-11T23:31:17.231+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:51058f37:::obj394:head local-lis/les=109/110 n=197 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:5170da72:::obj410:head) 2026-02-11T23:31:17.231+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.231+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5170da72:::obj410:head) v3 finish latency 0.000023 2026-02-11T23:31:17.231+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.231+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.231+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.231+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f2c000 con 0x3752000 2026-02-11T23:31:17.231+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:5178a392:::obj281:head)]) -- 0x4a05c80 con 0x3752000 2026-02-11T23:31:17.231+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.232+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 435 ==== MOSDPGPush(1.0 111/109 [PushOp(1:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43e56c0 con 0x3752000 2026-02-11T23:31:17.232+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.232+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.232+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 436 ==== pg_backfill(progress 1.0 e 111/111 lb 1:5178a392:::obj281:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f28580 con 0x3752000 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.232+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:5178a392:::obj281:head) v3 prio 15 type 95 cost 0 latency 0.000016 epoch 111 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 111/109 [PushOp(1:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.232+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5178a392:::obj281:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5178a392:::obj281:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:52d1d495:::obj578:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:52d1d495:::obj578:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:52d1d495:::obj578:head 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:52d1d495:::obj578:head v 107'578 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.232+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5178a392:::obj281:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.232+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5178a392:::obj281:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 111/111 lb 1:5178a392:::obj281:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.232+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.232+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:5178a392:::obj281:head) v3 2026-02-11T23:31:17.232+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5170da72:::obj410:head local-lis/les=109/110 n=198 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:5178a392:::obj281:head) 2026-02-11T23:31:17.232+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.232+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5178a392:::obj281:head) v3 finish latency 0.000018 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.232+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f2c480 con 0x3752000 2026-02-11T23:31:17.232+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:52d1d495:::obj578:head)]) -- 0x4f2c180 con 0x3752000 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.232+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 437 ==== MOSDPGPush(1.0 111/109 [PushOp(1:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea4e0 con 0x3752000 2026-02-11T23:31:17.232+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000017 epoch 111 2026-02-11T23:31:17.232+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.232+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 438 ==== pg_backfill(progress 1.0 e 111/111 lb 1:52d1d495:::obj578:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f28b00 con 0x3752000 2026-02-11T23:31:17.232+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:52d1d495:::obj578:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.232+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:52d1d495:::obj578:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 111/109 [PushOp(1:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:52d1d495:::obj578:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:52e063dd:::obj399:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:52e063dd:::obj399:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:52e063dd:::obj399:head 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:52e063dd:::obj399:head v 107'399 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.232+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:52d1d495:::obj578:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.232+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:52d1d495:::obj578:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 111/111 lb 1:52d1d495:::obj578:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.232+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.232+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:52d1d495:::obj578:head) v3 2026-02-11T23:31:17.232+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5178a392:::obj281:head local-lis/les=109/110 n=199 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:52d1d495:::obj578:head) 2026-02-11T23:31:17.232+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.232+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:52d1d495:::obj578:head) v3 finish latency 0.000013 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.232+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.232+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.232+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f2c900 con 0x3752000 2026-02-11T23:31:17.232+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:52e063dd:::obj399:head)]) -- 0x4f2c600 con 0x3752000 2026-02-11T23:31:17.232+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.233+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 439 ==== MOSDPGPush(1.0 111/109 [PushOp(1:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea9c0 con 0x3752000 2026-02-11T23:31:17.233+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 111 2026-02-11T23:31:17.233+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.233+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 440 ==== pg_backfill(progress 1.0 e 111/111 lb 1:52e063dd:::obj399:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f29080 con 0x3752000 2026-02-11T23:31:17.233+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:52e063dd:::obj399:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 111 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.233+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:52e063dd:::obj399:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.233+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 111/109 [PushOp(1:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.233+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.233+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.233+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.233+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.233+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:52e063dd:::obj399:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.233+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:53438dcf:::obj478:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:53438dcf:::obj478:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.233+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:53438dcf:::obj478:head 2026-02-11T23:31:17.233+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:53438dcf:::obj478:head v 107'478 2026-02-11T23:31:17.233+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.233+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.233+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.233+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.233+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:52e063dd:::obj399:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.233+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:52e063dd:::obj399:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 111/111 lb 1:52e063dd:::obj399:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.233+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.233+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:52e063dd:::obj399:head) v3 2026-02-11T23:31:17.233+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52d1d495:::obj578:head local-lis/les=109/110 n=200 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:52e063dd:::obj399:head) 2026-02-11T23:31:17.233+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.233+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:52e063dd:::obj399:head) v3 finish latency 0.000024 2026-02-11T23:31:17.233+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.233+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.233+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.233+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f2cd80 con 0x3752000 2026-02-11T23:31:17.233+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:53438dcf:::obj478:head)]) -- 0x4f2ca80 con 0x3752000 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.233+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.233+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.234+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 441 ==== MOSDPGPush(1.0 111/109 [PushOp(1:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43e4b60 con 0x3752000 2026-02-11T23:31:17.234+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000017 epoch 111 2026-02-11T23:31:17.234+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.234+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 442 ==== pg_backfill(progress 1.0 e 111/111 lb 1:53438dcf:::obj478:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f29600 con 0x3752000 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.234+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:53438dcf:::obj478:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 111 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.234+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:53438dcf:::obj478:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 111/109 [PushOp(1:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:53438dcf:::obj478:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:53c971fd:::obj132:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:53c971fd:::obj132:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:53c971fd:::obj132:head 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:53c971fd:::obj132:head v 107'132 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.234+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:53438dcf:::obj478:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:53438dcf:::obj478:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 111/111 lb 1:53438dcf:::obj478:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:53438dcf:::obj478:head) v3 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:52e063dd:::obj399:head local-lis/les=109/110 n=201 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:53438dcf:::obj478:head) 2026-02-11T23:31:17.234+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:53438dcf:::obj478:head) v3 finish latency 0.000028 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.234+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f2d200 con 0x3752000 2026-02-11T23:31:17.234+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:53c971fd:::obj132:head)]) -- 0x4f2cf00 con 0x3752000 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.234+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 443 ==== MOSDPGPush(1.0 111/109 [PushOp(1:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43ea000 con 0x3752000 2026-02-11T23:31:17.234+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000021 epoch 111 2026-02-11T23:31:17.234+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.234+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 444 ==== pg_backfill(progress 1.0 e 111/111 lb 1:53c971fd:::obj132:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f29b80 con 0x3752000 2026-02-11T23:31:17.234+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:53c971fd:::obj132:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.234+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:53c971fd:::obj132:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000054 MOSDPGPush(1.0 111/109 [PushOp(1:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:53c971fd:::obj132:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:541e28f4:::obj33:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:541e28f4:::obj33:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:541e28f4:::obj33:head 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:541e28f4:::obj33:head v 107'33 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.234+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.234+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:53c971fd:::obj132:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:53c971fd:::obj132:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 1.0 e 111/111 lb 1:53c971fd:::obj132:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:53c971fd:::obj132:head) v3 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53438dcf:::obj478:head local-lis/les=109/110 n=202 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:53c971fd:::obj132:head) 2026-02-11T23:31:17.234+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:53c971fd:::obj132:head) v3 finish latency 0.000015 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.234+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.234+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.234+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f2d380 con 0x3752000 2026-02-11T23:31:17.234+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:541e28f4:::obj33:head)]) -- 0x49b9c80 con 0x3752000 2026-02-11T23:31:17.234+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.235+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 445 ==== MOSDPGPush(1.0 111/109 [PushOp(1:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434c000 con 0x3752000 2026-02-11T23:31:17.235+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000020 epoch 111 2026-02-11T23:31:17.235+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.235+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.235+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 446 ==== pg_backfill(progress 1.0 e 111/111 lb 1:541e28f4:::obj33:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f2a100 con 0x3752000 2026-02-11T23:31:17.235+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:541e28f4:::obj33:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.235+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:541e28f4:::obj33:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.235+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.235+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.235+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.235+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.235+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 111/109 [PushOp(1:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.235+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.235+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:541e28f4:::obj33:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.235+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.235+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.235+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:545db4bf:::obj139:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.235+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:545db4bf:::obj139:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.235+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:545db4bf:::obj139:head 2026-02-11T23:31:17.235+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:545db4bf:::obj139:head v 107'139 2026-02-11T23:31:17.235+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.235+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.235+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.235+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.235+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.235+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.235+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:541e28f4:::obj33:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.235+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:541e28f4:::obj33:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 111/111 lb 1:541e28f4:::obj33:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.235+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.235+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:541e28f4:::obj33:head) v3 2026-02-11T23:31:17.235+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:53c971fd:::obj132:head local-lis/les=109/110 n=203 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:541e28f4:::obj33:head) 2026-02-11T23:31:17.235+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.235+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:541e28f4:::obj33:head) v3 finish latency 0.000022 2026-02-11T23:31:17.235+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.235+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.235+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.235+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.235+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.235+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.235+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f2d800 con 0x3752000 2026-02-11T23:31:17.235+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:545db4bf:::obj139:head)]) -- 0x4f2d500 con 0x3752000 2026-02-11T23:31:17.235+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.237+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 447 ==== MOSDPGPush(1.0 111/109 [PushOp(1:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eaea0 con 0x3752000 2026-02-11T23:31:17.237+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 111 2026-02-11T23:31:17.237+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.237+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 448 ==== pg_backfill(progress 1.0 e 111/111 lb 1:545db4bf:::obj139:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f2a680 con 0x3752000 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.237+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:545db4bf:::obj139:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.237+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:545db4bf:::obj139:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 111/109 [PushOp(1:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:545db4bf:::obj139:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:54a9e115:::obj404:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:54a9e115:::obj404:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:54a9e115:::obj404:head 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:54a9e115:::obj404:head v 107'404 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.237+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:545db4bf:::obj139:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.237+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:545db4bf:::obj139:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 111/111 lb 1:545db4bf:::obj139:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.237+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.237+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:545db4bf:::obj139:head) v3 2026-02-11T23:31:17.237+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:541e28f4:::obj33:head local-lis/les=109/110 n=204 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:545db4bf:::obj139:head) 2026-02-11T23:31:17.237+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.237+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:545db4bf:::obj139:head) v3 finish latency 0.000018 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.237+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f2dc80 con 0x3752000 2026-02-11T23:31:17.237+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:54a9e115:::obj404:head)]) -- 0x4f2d980 con 0x3752000 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.237+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 449 ==== MOSDPGPush(1.0 111/109 [PushOp(1:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eb520 con 0x3752000 2026-02-11T23:31:17.237+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000026 epoch 111 2026-02-11T23:31:17.237+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.237+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 450 ==== pg_backfill(progress 1.0 e 111/111 lb 1:54a9e115:::obj404:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f2ac00 con 0x3752000 2026-02-11T23:31:17.237+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:54a9e115:::obj404:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 111 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.237+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:54a9e115:::obj404:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000051 MOSDPGPush(1.0 111/109 [PushOp(1:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:54a9e115:::obj404:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:54cd4daa:::obj313:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:54cd4daa:::obj313:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:54cd4daa:::obj313:head 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:54cd4daa:::obj313:head v 107'313 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.237+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:54a9e115:::obj404:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.237+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:54a9e115:::obj404:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 111/111 lb 1:54a9e115:::obj404:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.237+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.237+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:54a9e115:::obj404:head) v3 2026-02-11T23:31:17.237+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:545db4bf:::obj139:head local-lis/les=109/110 n=205 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:54a9e115:::obj404:head) 2026-02-11T23:31:17.237+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.237+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:54a9e115:::obj404:head) v3 finish latency 0.000016 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.237+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.237+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.237+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f50180 con 0x3752000 2026-02-11T23:31:17.237+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:54cd4daa:::obj313:head)]) -- 0x4f2de00 con 0x3752000 2026-02-11T23:31:17.237+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.238+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 451 ==== MOSDPGPush(1.0 111/109 [PushOp(1:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3752000 2026-02-11T23:31:17.238+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 111 2026-02-11T23:31:17.238+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.238+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.238+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 452 ==== pg_backfill(progress 1.0 e 111/111 lb 1:54cd4daa:::obj313:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f2b180 con 0x3752000 2026-02-11T23:31:17.238+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.238+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:54cd4daa:::obj313:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 111 2026-02-11T23:31:17.238+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:54cd4daa:::obj313:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.238+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.238+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.238+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.238+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 111/109 [PushOp(1:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.238+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.238+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.238+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:54cd4daa:::obj313:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.238+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.238+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.238+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:554d3990:::obj276:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.238+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:554d3990:::obj276:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.238+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:554d3990:::obj276:head 2026-02-11T23:31:17.238+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:554d3990:::obj276:head v 107'276 2026-02-11T23:31:17.238+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.238+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.238+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000082 2026-02-11T23:31:17.238+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.238+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.238+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:54cd4daa:::obj313:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.238+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:54cd4daa:::obj313:head) v3 prio 15 cost 0 latency 0.000108 pg_backfill(progress 1.0 e 111/111 lb 1:54cd4daa:::obj313:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.238+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.238+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:54cd4daa:::obj313:head) v3 2026-02-11T23:31:17.238+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54a9e115:::obj404:head local-lis/les=109/110 n=206 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:54cd4daa:::obj313:head) 2026-02-11T23:31:17.238+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.238+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:54cd4daa:::obj313:head) v3 finish latency 0.000016 2026-02-11T23:31:17.238+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.238+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.238+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.238+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.238+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.238+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.238+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.238+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f50300 con 0x3752000 2026-02-11T23:31:17.238+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:554d3990:::obj276:head)]) -- 0x49b9e00 con 0x3752000 2026-02-11T23:31:17.238+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.239+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 453 ==== MOSDPGPush(1.0 111/109 [PushOp(1:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d1e0 con 0x3752000 2026-02-11T23:31:17.239+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 111 2026-02-11T23:31:17.239+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.239+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 454 ==== pg_backfill(progress 1.0 e 111/111 lb 1:554d3990:::obj276:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f2b700 con 0x3752000 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.239+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:554d3990:::obj276:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 111 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.239+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:554d3990:::obj276:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 111/109 [PushOp(1:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:554d3990:::obj276:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:55c8626c:::obj516:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:55c8626c:::obj516:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:55c8626c:::obj516:head 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:55c8626c:::obj516:head v 107'516 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.239+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:554d3990:::obj276:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:554d3990:::obj276:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 111/111 lb 1:554d3990:::obj276:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:554d3990:::obj276:head) v3 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:54cd4daa:::obj313:head local-lis/les=109/110 n=207 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:554d3990:::obj276:head) 2026-02-11T23:31:17.239+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:554d3990:::obj276:head) v3 finish latency 0.000023 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.239+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f50780 con 0x3752000 2026-02-11T23:31:17.239+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:55c8626c:::obj516:head)]) -- 0x4f50480 con 0x3752000 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.239+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 455 ==== MOSDPGPush(1.0 111/109 [PushOp(1:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d6c0 con 0x3752000 2026-02-11T23:31:17.239+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000019 epoch 111 2026-02-11T23:31:17.239+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.239+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 456 ==== pg_backfill(progress 1.0 e 111/111 lb 1:55c8626c:::obj516:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3752000 2026-02-11T23:31:17.239+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:55c8626c:::obj516:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 111 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.239+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:55c8626c:::obj516:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000052 MOSDPGPush(1.0 111/109 [PushOp(1:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:55c8626c:::obj516:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:55d2244d:::obj498:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:55d2244d:::obj498:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:55d2244d:::obj498:head 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:55d2244d:::obj498:head v 107'498 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.239+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.239+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.239+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:55c8626c:::obj516:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:55c8626c:::obj516:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 1.0 e 111/111 lb 1:55c8626c:::obj516:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:55c8626c:::obj516:head) v3 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:554d3990:::obj276:head local-lis/les=109/110 n=208 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:55c8626c:::obj516:head) 2026-02-11T23:31:17.239+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.239+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:55c8626c:::obj516:head) v3 finish latency 0.000015 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.239+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.240+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.240+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.240+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f50900 con 0x3752000 2026-02-11T23:31:17.240+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:55d2244d:::obj498:head)]) -- 0x4f52180 con 0x3752000 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.240+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 457 ==== MOSDPGPush(1.0 111/109 [PushOp(1:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434cb60 con 0x3752000 2026-02-11T23:31:17.240+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 111 2026-02-11T23:31:17.240+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.240+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 458 ==== pg_backfill(progress 1.0 e 111/111 lb 1:55d2244d:::obj498:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3752000 2026-02-11T23:31:17.240+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:55d2244d:::obj498:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 111 2026-02-11T23:31:17.240+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:55d2244d:::obj498:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.240+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 111/109 [PushOp(1:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.240+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.240+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.240+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.240+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:55d2244d:::obj498:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.240+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.240+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:565a3fbf:::obj474:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:565a3fbf:::obj474:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.240+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:565a3fbf:::obj474:head 2026-02-11T23:31:17.240+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:565a3fbf:::obj474:head v 107'474 2026-02-11T23:31:17.240+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.240+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.240+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.240+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.240+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:55d2244d:::obj498:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.240+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:55d2244d:::obj498:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 111/111 lb 1:55d2244d:::obj498:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.240+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.240+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:55d2244d:::obj498:head) v3 2026-02-11T23:31:17.240+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55c8626c:::obj516:head local-lis/les=109/110 n=209 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:55d2244d:::obj498:head) 2026-02-11T23:31:17.240+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.240+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:55d2244d:::obj498:head) v3 finish latency 0.000023 2026-02-11T23:31:17.240+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.240+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.240+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.240+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f50d80 con 0x3752000 2026-02-11T23:31:17.240+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:565a3fbf:::obj474:head)]) -- 0x4f50a80 con 0x3752000 2026-02-11T23:31:17.240+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.241+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 459 ==== MOSDPGPush(1.0 111/109 [PushOp(1:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d040 con 0x3752000 2026-02-11T23:31:17.241+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 111 2026-02-11T23:31:17.241+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.241+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 460 ==== pg_backfill(progress 1.0 e 111/111 lb 1:565a3fbf:::obj474:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3752000 2026-02-11T23:31:17.241+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:565a3fbf:::obj474:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 111 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.241+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:565a3fbf:::obj474:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:565a3fbf:::obj474:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000054 MOSDPGPush(1.0 111/109 [PushOp(1:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:5695f913:::obj241:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:5695f913:::obj241:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:5695f913:::obj241:head 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:5695f913:::obj241:head v 107'241 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.241+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:565a3fbf:::obj474:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:565a3fbf:::obj474:head) v3 prio 15 cost 0 latency 0.000103 pg_backfill(progress 1.0 e 111/111 lb 1:565a3fbf:::obj474:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:565a3fbf:::obj474:head) v3 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:55d2244d:::obj498:head local-lis/les=109/110 n=210 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:565a3fbf:::obj474:head) 2026-02-11T23:31:17.241+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:565a3fbf:::obj474:head) v3 finish latency 0.000015 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.241+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f50f00 con 0x3752000 2026-02-11T23:31:17.241+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:5695f913:::obj241:head)]) -- 0x4f52480 con 0x3752000 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.241+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 461 ==== MOSDPGPush(1.0 111/109 [PushOp(1:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434c4e0 con 0x3752000 2026-02-11T23:31:17.241+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000019 epoch 111 2026-02-11T23:31:17.241+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.241+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 462 ==== pg_backfill(progress 1.0 e 111/111 lb 1:5695f913:::obj241:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5b080 con 0x3752000 2026-02-11T23:31:17.241+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:5695f913:::obj241:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.241+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5695f913:::obj241:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000048 MOSDPGPush(1.0 111/109 [PushOp(1:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5695f913:::obj241:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:56fc6216:::obj287:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:56fc6216:::obj287:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:56fc6216:::obj287:head 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:56fc6216:::obj287:head v 107'287 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.241+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5695f913:::obj241:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5695f913:::obj241:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 111/111 lb 1:5695f913:::obj241:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:5695f913:::obj241:head) v3 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:565a3fbf:::obj474:head local-lis/les=109/110 n=211 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:5695f913:::obj241:head) 2026-02-11T23:31:17.241+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.241+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5695f913:::obj241:head) v3 finish latency 0.000022 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.241+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.241+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.241+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f51380 con 0x3752000 2026-02-11T23:31:17.241+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:56fc6216:::obj287:head)]) -- 0x4f51080 con 0x3752000 2026-02-11T23:31:17.241+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.242+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.242+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 463 ==== MOSDPGPush(1.0 111/109 [PushOp(1:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4445d40 con 0x3752000 2026-02-11T23:31:17.242+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 111 2026-02-11T23:31:17.242+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.242+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.242+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 464 ==== pg_backfill(progress 1.0 e 111/111 lb 1:56fc6216:::obj287:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5b600 con 0x3752000 2026-02-11T23:31:17.242+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.242+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:56fc6216:::obj287:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 111 2026-02-11T23:31:17.242+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:56fc6216:::obj287:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.242+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.242+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.242+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 111/109 [PushOp(1:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.242+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.242+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:56fc6216:::obj287:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.242+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.242+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:578304e3:::obj249:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.242+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:578304e3:::obj249:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.242+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:578304e3:::obj249:head 2026-02-11T23:31:17.242+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:578304e3:::obj249:head v 107'249 2026-02-11T23:31:17.242+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.242+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.242+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:17.242+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:56fc6216:::obj287:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.242+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:56fc6216:::obj287:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 1.0 e 111/111 lb 1:56fc6216:::obj287:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.242+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.242+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:56fc6216:::obj287:head) v3 2026-02-11T23:31:17.242+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5695f913:::obj241:head local-lis/les=109/110 n=212 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:56fc6216:::obj287:head) 2026-02-11T23:31:17.242+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.242+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:56fc6216:::obj287:head) v3 finish latency 0.000016 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.242+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.242+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.242+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f51500 con 0x3752000 2026-02-11T23:31:17.242+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:578304e3:::obj249:head)]) -- 0x4f52780 con 0x3752000 2026-02-11T23:31:17.242+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.243+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 465 ==== MOSDPGPush(1.0 111/109 [PushOp(1:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2680 con 0x3752000 2026-02-11T23:31:17.243+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000024 epoch 111 2026-02-11T23:31:17.243+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.243+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 466 ==== pg_backfill(progress 1.0 e 111/111 lb 1:578304e3:::obj249:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5bb80 con 0x3752000 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.243+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:578304e3:::obj249:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 111 2026-02-11T23:31:17.243+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:578304e3:::obj249:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000059 MOSDPGPush(1.0 111/109 [PushOp(1:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:578304e3:::obj249:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:57ba9973:::obj296:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:57ba9973:::obj296:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:57ba9973:::obj296:head 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:57ba9973:::obj296:head v 107'296 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.243+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:578304e3:::obj249:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.243+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:578304e3:::obj249:head) v3 prio 15 cost 0 latency 0.000099 pg_backfill(progress 1.0 e 111/111 lb 1:578304e3:::obj249:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.243+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.243+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:578304e3:::obj249:head) v3 2026-02-11T23:31:17.243+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:56fc6216:::obj287:head local-lis/les=109/110 n=213 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:578304e3:::obj249:head) 2026-02-11T23:31:17.243+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.243+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:578304e3:::obj249:head) v3 finish latency 0.000013 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.243+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.243+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f51680 con 0x3752000 2026-02-11T23:31:17.243+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:57ba9973:::obj296:head)]) -- 0x4f52a80 con 0x3752000 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.243+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 467 ==== MOSDPGPush(1.0 111/109 [PushOp(1:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x434da00 con 0x3752000 2026-02-11T23:31:17.243+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 111 2026-02-11T23:31:17.243+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.243+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 468 ==== pg_backfill(progress 1.0 e 111/111 lb 1:57ba9973:::obj296:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3752000 2026-02-11T23:31:17.243+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:57ba9973:::obj296:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.243+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:57ba9973:::obj296:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 111/109 [PushOp(1:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:57ba9973:::obj296:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:5880e7a7:::obj64:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:5880e7a7:::obj64:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:5880e7a7:::obj64:head 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:5880e7a7:::obj64:head v 107'64 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.243+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.243+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:17.243+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:57ba9973:::obj296:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.243+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:57ba9973:::obj296:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 111/111 lb 1:57ba9973:::obj296:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.243+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.243+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:57ba9973:::obj296:head) v3 2026-02-11T23:31:17.243+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:578304e3:::obj249:head local-lis/les=109/110 n=214 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:57ba9973:::obj296:head) 2026-02-11T23:31:17.243+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.243+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:57ba9973:::obj296:head) v3 finish latency 0.000013 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.243+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.244+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.244+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.244+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f51800 con 0x3752000 2026-02-11T23:31:17.244+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:5880e7a7:::obj64:head)]) -- 0x4f52d80 con 0x3752000 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.244+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 469 ==== MOSDPGPush(1.0 111/109 [PushOp(1:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3752000 2026-02-11T23:31:17.244+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000019 epoch 111 2026-02-11T23:31:17.244+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.244+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.244+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 470 ==== pg_backfill(progress 1.0 e 111/111 lb 1:5880e7a7:::obj64:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3752000 2026-02-11T23:31:17.244+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.244+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:5880e7a7:::obj64:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 111 2026-02-11T23:31:17.244+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5880e7a7:::obj64:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.244+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.244+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.244+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000052 MOSDPGPush(1.0 111/109 [PushOp(1:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.244+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.244+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5880e7a7:::obj64:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.244+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.244+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:58980d3a:::obj339:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.244+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:58980d3a:::obj339:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.244+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:58980d3a:::obj339:head 2026-02-11T23:31:17.244+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:58980d3a:::obj339:head v 107'339 2026-02-11T23:31:17.244+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.244+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.244+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:17.244+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5880e7a7:::obj64:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.244+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5880e7a7:::obj64:head) v3 prio 15 cost 0 latency 0.000100 pg_backfill(progress 1.0 e 111/111 lb 1:5880e7a7:::obj64:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.244+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.244+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:5880e7a7:::obj64:head) v3 2026-02-11T23:31:17.244+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:57ba9973:::obj296:head local-lis/les=109/110 n=215 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:5880e7a7:::obj64:head) 2026-02-11T23:31:17.244+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.244+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5880e7a7:::obj64:head) v3 finish latency 0.000014 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.244+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.244+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.244+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f51980 con 0x3752000 2026-02-11T23:31:17.244+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:58980d3a:::obj339:head)]) -- 0x4f52f00 con 0x3752000 2026-02-11T23:31:17.244+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.245+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 471 ==== MOSDPGPush(1.0 111/109 [PushOp(1:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4445380 con 0x3752000 2026-02-11T23:31:17.245+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.245+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.245+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 472 ==== pg_backfill(progress 1.0 e 111/111 lb 1:58980d3a:::obj339:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3752000 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.245+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:58980d3a:::obj339:head) v3 prio 15 type 95 cost 0 latency 0.000014 epoch 111 2026-02-11T23:31:17.245+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:58980d3a:::obj339:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 111/109 [PushOp(1:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:58980d3a:::obj339:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:5946d48f:::obj257:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:5946d48f:::obj257:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:5946d48f:::obj257:head 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:5946d48f:::obj257:head v 107'257 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.245+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:58980d3a:::obj339:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:58980d3a:::obj339:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 1.0 e 111/111 lb 1:58980d3a:::obj339:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:58980d3a:::obj339:head) v3 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5880e7a7:::obj64:head local-lis/les=109/110 n=216 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:58980d3a:::obj339:head) 2026-02-11T23:31:17.245+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:58980d3a:::obj339:head) v3 finish latency 0.000012 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.245+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f51b00 con 0x3752000 2026-02-11T23:31:17.245+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:5946d48f:::obj257:head)]) -- 0x4f53200 con 0x3752000 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.245+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 473 ==== MOSDPGPush(1.0 111/109 [PushOp(1:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4445860 con 0x3752000 2026-02-11T23:31:17.245+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 111 2026-02-11T23:31:17.245+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.245+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 474 ==== pg_backfill(progress 1.0 e 111/111 lb 1:5946d48f:::obj257:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3752000 2026-02-11T23:31:17.245+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:5946d48f:::obj257:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 111 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.245+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5946d48f:::obj257:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5946d48f:::obj257:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000058 MOSDPGPush(1.0 111/109 [PushOp(1:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:5a01aba7:::obj211:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:5a01aba7:::obj211:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:5a01aba7:::obj211:head 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:5a01aba7:::obj211:head v 107'211 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.245+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.245+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.245+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5946d48f:::obj257:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5946d48f:::obj257:head) v3 prio 15 cost 0 latency 0.000106 pg_backfill(progress 1.0 e 111/111 lb 1:5946d48f:::obj257:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:5946d48f:::obj257:head) v3 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:58980d3a:::obj339:head local-lis/les=109/110 n=217 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:5946d48f:::obj257:head) 2026-02-11T23:31:17.245+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.245+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5946d48f:::obj257:head) v3 finish latency 0.000022 2026-02-11T23:31:17.245+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.246+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.246+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.246+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6e000 con 0x3752000 2026-02-11T23:31:17.246+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:5a01aba7:::obj211:head)]) -- 0x4f51c80 con 0x3752000 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.246+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 475 ==== MOSDPGPush(1.0 111/109 [PushOp(1:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44449c0 con 0x3752000 2026-02-11T23:31:17.246+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.246+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.246+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.246+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.246+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 476 ==== pg_backfill(progress 1.0 e 111/111 lb 1:5a01aba7:::obj211:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d700 con 0x3752000 2026-02-11T23:31:17.246+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.246+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:5a01aba7:::obj211:head) v3 prio 15 type 95 cost 0 latency 0.000015 epoch 111 2026-02-11T23:31:17.246+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5a01aba7:::obj211:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.246+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.246+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 111/109 [PushOp(1:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.246+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.246+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5a01aba7:::obj211:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.246+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.246+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:5a322cd1:::obj402:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.246+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:5a322cd1:::obj402:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.246+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:5a322cd1:::obj402:head 2026-02-11T23:31:17.246+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:5a322cd1:::obj402:head v 107'402 2026-02-11T23:31:17.246+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.246+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.246+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.246+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5a01aba7:::obj211:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.246+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5a01aba7:::obj211:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 1.0 e 111/111 lb 1:5a01aba7:::obj211:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.246+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.246+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:5a01aba7:::obj211:head) v3 2026-02-11T23:31:17.246+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5946d48f:::obj257:head local-lis/les=109/110 n=218 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:5a01aba7:::obj211:head) 2026-02-11T23:31:17.246+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.246+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5a01aba7:::obj211:head) v3 finish latency 0.000016 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.246+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.246+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.246+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6e180 con 0x3752000 2026-02-11T23:31:17.246+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:5a322cd1:::obj402:head)]) -- 0x4f53500 con 0x3752000 2026-02-11T23:31:17.246+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.247+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 477 ==== MOSDPGPush(1.0 111/109 [PushOp(1:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44ba1a0 con 0x3752000 2026-02-11T23:31:17.247+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 111 2026-02-11T23:31:17.247+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.247+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 478 ==== pg_backfill(progress 1.0 e 111/111 lb 1:5a322cd1:::obj402:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f70000 con 0x3752000 2026-02-11T23:31:17.247+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:5a322cd1:::obj402:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 111 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.247+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5a322cd1:::obj402:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 111/109 [PushOp(1:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5a322cd1:::obj402:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:5a3c6224:::obj352:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:5a3c6224:::obj352:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:5a3c6224:::obj352:head 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:5a3c6224:::obj352:head v 107'352 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.247+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5a322cd1:::obj402:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5a322cd1:::obj402:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 111/111 lb 1:5a322cd1:::obj402:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:5a322cd1:::obj402:head) v3 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a01aba7:::obj211:head local-lis/les=109/110 n=219 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:5a322cd1:::obj402:head) 2026-02-11T23:31:17.247+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5a322cd1:::obj402:head) v3 finish latency 0.000023 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.247+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6e600 con 0x3752000 2026-02-11T23:31:17.247+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:5a3c6224:::obj352:head)]) -- 0x4f6e300 con 0x3752000 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.247+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 479 ==== MOSDPGPush(1.0 111/109 [PushOp(1:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44ba680 con 0x3752000 2026-02-11T23:31:17.247+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 111 2026-02-11T23:31:17.247+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.247+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 480 ==== pg_backfill(progress 1.0 e 111/111 lb 1:5a3c6224:::obj352:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f70580 con 0x3752000 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.247+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:5a3c6224:::obj352:head) v3 prio 15 type 95 cost 0 latency 0.000018 epoch 111 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.247+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5a3c6224:::obj352:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000053 MOSDPGPush(1.0 111/109 [PushOp(1:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5a3c6224:::obj352:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:5d17b8a1:::obj341:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:5d17b8a1:::obj341:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:5d17b8a1:::obj341:head 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:5d17b8a1:::obj341:head v 107'341 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.247+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.247+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5a3c6224:::obj352:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5a3c6224:::obj352:head) v3 prio 15 cost 0 latency 0.000101 pg_backfill(progress 1.0 e 111/111 lb 1:5a3c6224:::obj352:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:5a3c6224:::obj352:head) v3 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a322cd1:::obj402:head local-lis/les=109/110 n=220 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:5a3c6224:::obj352:head) 2026-02-11T23:31:17.247+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5a3c6224:::obj352:head) v3 finish latency 0.000015 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.247+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.247+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.247+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6e780 con 0x3752000 2026-02-11T23:31:17.247+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:5d17b8a1:::obj341:head)]) -- 0x4f53800 con 0x3752000 2026-02-11T23:31:17.247+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.248+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 481 ==== MOSDPGPush(1.0 111/109 [PushOp(1:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44af6c0 con 0x3752000 2026-02-11T23:31:17.248+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 111 2026-02-11T23:31:17.248+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.248+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.248+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 482 ==== pg_backfill(progress 1.0 e 111/111 lb 1:5d17b8a1:::obj341:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f70b00 con 0x3752000 2026-02-11T23:31:17.248+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:5d17b8a1:::obj341:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 111 2026-02-11T23:31:17.248+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.248+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5d17b8a1:::obj341:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.248+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.248+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.248+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 111/109 [PushOp(1:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.248+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.248+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.248+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.248+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5d17b8a1:::obj341:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.248+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.248+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:5e27c68c:::obj582:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.248+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:5e27c68c:::obj582:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.248+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:5e27c68c:::obj582:head 2026-02-11T23:31:17.248+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:5e27c68c:::obj582:head v 107'582 2026-02-11T23:31:17.248+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.248+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.248+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.248+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.248+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.248+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.248+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5d17b8a1:::obj341:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.248+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5d17b8a1:::obj341:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 111/111 lb 1:5d17b8a1:::obj341:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.248+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.248+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:5d17b8a1:::obj341:head) v3 2026-02-11T23:31:17.248+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5a3c6224:::obj352:head local-lis/les=109/110 n=221 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:5d17b8a1:::obj341:head) 2026-02-11T23:31:17.248+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.248+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5d17b8a1:::obj341:head) v3 finish latency 0.000021 2026-02-11T23:31:17.248+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.248+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.248+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.248+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.248+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.248+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.248+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6ec00 con 0x3752000 2026-02-11T23:31:17.248+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:5e27c68c:::obj582:head)]) -- 0x4f6e900 con 0x3752000 2026-02-11T23:31:17.248+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.249+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 483 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44afba0 con 0x3752000 2026-02-11T23:31:17.249+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 111 2026-02-11T23:31:17.249+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.249+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 484 ==== pg_backfill(progress 1.0 e 111/111 lb 1:5e27c68c:::obj582:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f71080 con 0x3752000 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.249+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:5e27c68c:::obj582:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 111 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.249+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5e27c68c:::obj582:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 111/109 [PushOp(1:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5e27c68c:::obj582:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6131e0b1:::obj290:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6131e0b1:::obj290:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6131e0b1:::obj290:head 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6131e0b1:::obj290:head v 107'290 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.249+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:5e27c68c:::obj582:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.249+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5e27c68c:::obj582:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 111/111 lb 1:5e27c68c:::obj582:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.249+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.249+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:5e27c68c:::obj582:head) v3 2026-02-11T23:31:17.249+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5d17b8a1:::obj341:head local-lis/les=109/110 n=222 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:5e27c68c:::obj582:head) 2026-02-11T23:31:17.249+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.249+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:5e27c68c:::obj582:head) v3 finish latency 0.000018 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.249+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6f080 con 0x3752000 2026-02-11T23:31:17.249+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6131e0b1:::obj290:head)]) -- 0x4f6ed80 con 0x3752000 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.249+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 485 ==== MOSDPGPush(1.0 111/109 [PushOp(1:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4604000 con 0x3752000 2026-02-11T23:31:17.249+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 111 2026-02-11T23:31:17.249+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.249+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 486 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6131e0b1:::obj290:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f71600 con 0x3752000 2026-02-11T23:31:17.249+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6131e0b1:::obj290:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 111 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.249+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6131e0b1:::obj290:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 111/109 [PushOp(1:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6131e0b1:::obj290:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:614752af:::obj331:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:614752af:::obj331:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:614752af:::obj331:head 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:614752af:::obj331:head v 107'331 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.249+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6131e0b1:::obj290:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.249+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6131e0b1:::obj290:head) v3 prio 15 cost 0 latency 0.000099 pg_backfill(progress 1.0 e 111/111 lb 1:6131e0b1:::obj290:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.249+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.249+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6131e0b1:::obj290:head) v3 2026-02-11T23:31:17.249+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:5e27c68c:::obj582:head local-lis/les=109/110 n=223 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6131e0b1:::obj290:head) 2026-02-11T23:31:17.249+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.249+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6131e0b1:::obj290:head) v3 finish latency 0.000017 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.249+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.249+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.249+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6f500 con 0x3752000 2026-02-11T23:31:17.249+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:614752af:::obj331:head)]) -- 0x4f6f200 con 0x3752000 2026-02-11T23:31:17.249+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.250+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 487 ==== MOSDPGPush(1.0 111/109 [PushOp(1:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x46044e0 con 0x3752000 2026-02-11T23:31:17.250+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.250+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.250+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.250+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 488 ==== pg_backfill(progress 1.0 e 111/111 lb 1:614752af:::obj331:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f71b80 con 0x3752000 2026-02-11T23:31:17.250+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:614752af:::obj331:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.250+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:614752af:::obj331:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.250+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.250+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.250+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.250+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 111/109 [PushOp(1:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.250+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.250+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.250+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:614752af:::obj331:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.250+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.250+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.250+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:61741462:::obj327:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.250+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:61741462:::obj327:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.250+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:61741462:::obj327:head 2026-02-11T23:31:17.250+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:61741462:::obj327:head v 107'327 2026-02-11T23:31:17.250+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.250+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.250+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.250+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.250+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.250+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:614752af:::obj331:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.250+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:614752af:::obj331:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 111/111 lb 1:614752af:::obj331:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.250+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.250+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:614752af:::obj331:head) v3 2026-02-11T23:31:17.250+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6131e0b1:::obj290:head local-lis/les=109/110 n=224 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:614752af:::obj331:head) 2026-02-11T23:31:17.250+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.250+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:614752af:::obj331:head) v3 finish latency 0.000014 2026-02-11T23:31:17.250+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.250+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.250+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.250+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.250+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.250+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.250+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.250+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6f680 con 0x3752000 2026-02-11T23:31:17.250+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:61741462:::obj327:head)]) -- 0x4f53b00 con 0x3752000 2026-02-11T23:31:17.250+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.251+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 489 ==== MOSDPGPush(1.0 111/109 [PushOp(1:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x46049c0 con 0x3752000 2026-02-11T23:31:17.251+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000017 epoch 111 2026-02-11T23:31:17.251+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.251+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 490 ==== pg_backfill(progress 1.0 e 111/111 lb 1:61741462:::obj327:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f72100 con 0x3752000 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 111/109 [PushOp(1:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.251+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:61741462:::obj327:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 111 2026-02-11T23:31:17.251+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:61741462:::obj327:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:61741462:::obj327:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:61dc80df:::obj452:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:61dc80df:::obj452:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:61dc80df:::obj452:head 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:61dc80df:::obj452:head v 107'452 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.251+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:61741462:::obj327:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:61741462:::obj327:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 111/111 lb 1:61741462:::obj327:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:61741462:::obj327:head) v3 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:614752af:::obj331:head local-lis/les=109/110 n=225 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:61741462:::obj327:head) 2026-02-11T23:31:17.251+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:61741462:::obj327:head) v3 finish latency 0.000019 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.251+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6f800 con 0x3752000 2026-02-11T23:31:17.251+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:61dc80df:::obj452:head)]) -- 0x4f53e00 con 0x3752000 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.251+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 491 ==== MOSDPGPush(1.0 111/109 [PushOp(1:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4604ea0 con 0x3752000 2026-02-11T23:31:17.251+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 111 2026-02-11T23:31:17.251+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.251+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 492 ==== pg_backfill(progress 1.0 e 111/111 lb 1:61dc80df:::obj452:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f72680 con 0x3752000 2026-02-11T23:31:17.251+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:61dc80df:::obj452:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 111 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.251+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:61dc80df:::obj452:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 111/109 [PushOp(1:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:61dc80df:::obj452:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:622f1330:::obj230:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:622f1330:::obj230:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:622f1330:::obj230:head 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:622f1330:::obj230:head v 107'230 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.251+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.251+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:61dc80df:::obj452:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:61dc80df:::obj452:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 111/111 lb 1:61dc80df:::obj452:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:61dc80df:::obj452:head) v3 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61741462:::obj327:head local-lis/les=109/110 n=226 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:61dc80df:::obj452:head) 2026-02-11T23:31:17.251+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:61dc80df:::obj452:head) v3 finish latency 0.000020 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.251+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.251+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.251+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.251+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6f980 con 0x3752000 2026-02-11T23:31:17.251+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:622f1330:::obj230:head)]) -- 0x4f88180 con 0x3752000 2026-02-11T23:31:17.252+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.252+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 493 ==== MOSDPGPush(1.0 111/109 [PushOp(1:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x46024e0 con 0x3752000 2026-02-11T23:31:17.252+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.252+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.252+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.252+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 494 ==== pg_backfill(progress 1.0 e 111/111 lb 1:622f1330:::obj230:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f72c00 con 0x3752000 2026-02-11T23:31:17.252+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:622f1330:::obj230:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 111 2026-02-11T23:31:17.252+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.252+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:622f1330:::obj230:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.252+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.252+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.252+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.252+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 111/109 [PushOp(1:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.252+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:622f1330:::obj230:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.252+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.252+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.252+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.252+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:624e19c5:::obj464:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.252+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:624e19c5:::obj464:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.252+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:624e19c5:::obj464:head 2026-02-11T23:31:17.252+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:624e19c5:::obj464:head v 107'464 2026-02-11T23:31:17.252+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.252+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.252+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.252+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.252+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.252+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:622f1330:::obj230:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.252+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:622f1330:::obj230:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 111/111 lb 1:622f1330:::obj230:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.252+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.252+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:622f1330:::obj230:head) v3 2026-02-11T23:31:17.252+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:61dc80df:::obj452:head local-lis/les=109/110 n=227 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:622f1330:::obj230:head) 2026-02-11T23:31:17.252+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.252+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:622f1330:::obj230:head) v3 finish latency 0.000018 2026-02-11T23:31:17.252+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.252+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.252+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.252+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.252+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.252+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.252+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.252+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6fb00 con 0x3752000 2026-02-11T23:31:17.252+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:624e19c5:::obj464:head)]) -- 0x4f88480 con 0x3752000 2026-02-11T23:31:17.252+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.253+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 495 ==== MOSDPGPush(1.0 111/109 [PushOp(1:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x46029c0 con 0x3752000 2026-02-11T23:31:17.253+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 111 2026-02-11T23:31:17.253+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.253+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 496 ==== pg_backfill(progress 1.0 e 111/111 lb 1:624e19c5:::obj464:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f73180 con 0x3752000 2026-02-11T23:31:17.253+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:624e19c5:::obj464:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 111 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.253+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:624e19c5:::obj464:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 111/109 [PushOp(1:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:624e19c5:::obj464:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:62eb9ce0:::obj205:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:62eb9ce0:::obj205:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:62eb9ce0:::obj205:head 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:62eb9ce0:::obj205:head v 107'205 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.253+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:624e19c5:::obj464:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:624e19c5:::obj464:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 111/111 lb 1:624e19c5:::obj464:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:624e19c5:::obj464:head) v3 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:622f1330:::obj230:head local-lis/les=109/110 n=228 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:624e19c5:::obj464:head) 2026-02-11T23:31:17.253+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:624e19c5:::obj464:head) v3 finish latency 0.000013 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.253+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6fc80 con 0x3752000 2026-02-11T23:31:17.253+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:62eb9ce0:::obj205:head)]) -- 0x4f88780 con 0x3752000 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.253+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 497 ==== MOSDPGPush(1.0 111/109 [PushOp(1:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43e4340 con 0x3752000 2026-02-11T23:31:17.253+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 111 2026-02-11T23:31:17.253+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.253+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 498 ==== pg_backfill(progress 1.0 e 111/111 lb 1:62eb9ce0:::obj205:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f73700 con 0x3752000 2026-02-11T23:31:17.253+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:62eb9ce0:::obj205:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 111 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.253+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:62eb9ce0:::obj205:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 111/109 [PushOp(1:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:62eb9ce0:::obj205:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:63191178:::obj308:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:63191178:::obj308:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:63191178:::obj308:head 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:63191178:::obj308:head v 107'308 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.253+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.253+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:62eb9ce0:::obj205:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:62eb9ce0:::obj205:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 111/111 lb 1:62eb9ce0:::obj205:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:62eb9ce0:::obj205:head) v3 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:624e19c5:::obj464:head local-lis/les=109/110 n=229 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:62eb9ce0:::obj205:head) 2026-02-11T23:31:17.253+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:62eb9ce0:::obj205:head) v3 finish latency 0.000012 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.253+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.253+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.253+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4f6fe00 con 0x3752000 2026-02-11T23:31:17.253+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:63191178:::obj308:head)]) -- 0x4f88a80 con 0x3752000 2026-02-11T23:31:17.253+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.254+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 499 ==== MOSDPGPush(1.0 111/109 [PushOp(1:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4602000 con 0x3752000 2026-02-11T23:31:17.254+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 111 2026-02-11T23:31:17.254+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.254+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.254+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.254+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.254+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.254+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 500 ==== pg_backfill(progress 1.0 e 111/111 lb 1:63191178:::obj308:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f9c000 con 0x3752000 2026-02-11T23:31:17.254+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:63191178:::obj308:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 111 2026-02-11T23:31:17.254+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.254+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:63191178:::obj308:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.254+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 111/109 [PushOp(1:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.254+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.254+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.254+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:63191178:::obj308:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.254+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.254+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.254+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:640d3668:::obj286:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.254+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:640d3668:::obj286:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.254+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:640d3668:::obj286:head 2026-02-11T23:31:17.254+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:640d3668:::obj286:head v 107'286 2026-02-11T23:31:17.254+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.254+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.254+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.254+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.254+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.254+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:63191178:::obj308:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.254+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:63191178:::obj308:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 111/111 lb 1:63191178:::obj308:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.254+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.254+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:63191178:::obj308:head) v3 2026-02-11T23:31:17.254+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:62eb9ce0:::obj205:head local-lis/les=109/110 n=230 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:63191178:::obj308:head) 2026-02-11T23:31:17.254+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.254+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:63191178:::obj308:head) v3 finish latency 0.000016 2026-02-11T23:31:17.254+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.254+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.254+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.254+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.254+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.254+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.254+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.254+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa0000 con 0x3752000 2026-02-11T23:31:17.254+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:640d3668:::obj286:head)]) -- 0x4f88d80 con 0x3752000 2026-02-11T23:31:17.254+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.255+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 501 ==== MOSDPGPush(1.0 111/109 [PushOp(1:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4361ba0 con 0x3752000 2026-02-11T23:31:17.255+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000022 epoch 111 2026-02-11T23:31:17.255+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.255+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 502 ==== pg_backfill(progress 1.0 e 111/111 lb 1:640d3668:::obj286:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f9c580 con 0x3752000 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.255+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:640d3668:::obj286:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 111 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000057 MOSDPGPush(1.0 111/109 [PushOp(1:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.255+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:640d3668:::obj286:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:640d3668:::obj286:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:649aec78:::obj275:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:649aec78:::obj275:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:649aec78:::obj275:head 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:649aec78:::obj275:head v 107'275 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.255+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:640d3668:::obj286:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:640d3668:::obj286:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 111/111 lb 1:640d3668:::obj286:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:640d3668:::obj286:head) v3 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:63191178:::obj308:head local-lis/les=109/110 n=231 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:640d3668:::obj286:head) 2026-02-11T23:31:17.255+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:640d3668:::obj286:head) v3 finish latency 0.000017 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.255+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa0180 con 0x3752000 2026-02-11T23:31:17.255+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:649aec78:::obj275:head)]) -- 0x4f89080 con 0x3752000 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.255+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 503 ==== MOSDPGPush(1.0 111/109 [PushOp(1:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4360d00 con 0x3752000 2026-02-11T23:31:17.255+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 111 2026-02-11T23:31:17.255+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.255+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 504 ==== pg_backfill(progress 1.0 e 111/111 lb 1:649aec78:::obj275:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f9cb00 con 0x3752000 2026-02-11T23:31:17.255+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:649aec78:::obj275:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.255+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:649aec78:::obj275:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:649aec78:::obj275:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000051 MOSDPGPush(1.0 111/109 [PushOp(1:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:64b337fe:::obj71:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:64b337fe:::obj71:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:64b337fe:::obj71:head 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:64b337fe:::obj71:head v 107'71 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.255+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.255+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:649aec78:::obj275:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:649aec78:::obj275:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 111/111 lb 1:649aec78:::obj275:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:649aec78:::obj275:head) v3 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:640d3668:::obj286:head local-lis/les=109/110 n=232 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:649aec78:::obj275:head) 2026-02-11T23:31:17.255+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:649aec78:::obj275:head) v3 finish latency 0.000014 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.255+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.255+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.255+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa0300 con 0x3752000 2026-02-11T23:31:17.255+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:64b337fe:::obj71:head)]) -- 0x4f89380 con 0x3752000 2026-02-11T23:31:17.255+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.256+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 505 ==== MOSDPGPush(1.0 111/109 [PushOp(1:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43611e0 con 0x3752000 2026-02-11T23:31:17.256+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 111 2026-02-11T23:31:17.256+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.256+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.256+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 506 ==== pg_backfill(progress 1.0 e 111/111 lb 1:64b337fe:::obj71:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f9d080 con 0x3752000 2026-02-11T23:31:17.256+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:64b337fe:::obj71:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 111 2026-02-11T23:31:17.256+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.256+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:64b337fe:::obj71:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.256+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.256+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.256+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 111/109 [PushOp(1:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.256+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.256+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.256+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:64b337fe:::obj71:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.256+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.256+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.256+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:64bc0383:::obj214:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.256+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:64bc0383:::obj214:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.256+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:64bc0383:::obj214:head 2026-02-11T23:31:17.256+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:64bc0383:::obj214:head v 107'214 2026-02-11T23:31:17.256+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.256+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.256+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.256+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.256+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.256+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:64b337fe:::obj71:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.256+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:64b337fe:::obj71:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 111/111 lb 1:64b337fe:::obj71:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.256+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.256+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:64b337fe:::obj71:head) v3 2026-02-11T23:31:17.256+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:649aec78:::obj275:head local-lis/les=109/110 n=233 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:64b337fe:::obj71:head) 2026-02-11T23:31:17.256+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.256+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:64b337fe:::obj71:head) v3 finish latency 0.000012 2026-02-11T23:31:17.256+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.256+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.256+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.256+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.256+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.256+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.256+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.256+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa0480 con 0x3752000 2026-02-11T23:31:17.256+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:64bc0383:::obj214:head)]) -- 0x4f89500 con 0x3752000 2026-02-11T23:31:17.256+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.257+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 507 ==== MOSDPGPush(1.0 111/109 [PushOp(1:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43616c0 con 0x3752000 2026-02-11T23:31:17.257+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 111 2026-02-11T23:31:17.257+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.257+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 508 ==== pg_backfill(progress 1.0 e 111/111 lb 1:64bc0383:::obj214:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f9d600 con 0x3752000 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.257+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:64bc0383:::obj214:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.257+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:64bc0383:::obj214:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 111/109 [PushOp(1:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:64bc0383:::obj214:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:64c1dddc:::obj83:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:64c1dddc:::obj83:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:64c1dddc:::obj83:head 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:64c1dddc:::obj83:head v 107'83 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.257+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:64bc0383:::obj214:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:64bc0383:::obj214:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 111/111 lb 1:64bc0383:::obj214:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:64bc0383:::obj214:head) v3 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64b337fe:::obj71:head local-lis/les=109/110 n=234 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:64bc0383:::obj214:head) 2026-02-11T23:31:17.257+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:64bc0383:::obj214:head) v3 finish latency 0.000015 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.257+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa0600 con 0x3752000 2026-02-11T23:31:17.257+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:64c1dddc:::obj83:head)]) -- 0x4f89800 con 0x3752000 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.257+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 509 ==== MOSDPGPush(1.0 111/109 [PushOp(1:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43604e0 con 0x3752000 2026-02-11T23:31:17.257+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 111 2026-02-11T23:31:17.257+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.257+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 510 ==== pg_backfill(progress 1.0 e 111/111 lb 1:64c1dddc:::obj83:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f9db80 con 0x3752000 2026-02-11T23:31:17.257+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:64c1dddc:::obj83:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 111 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.257+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:64c1dddc:::obj83:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 111/109 [PushOp(1:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:64c1dddc:::obj83:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:657806de:::obj544:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:657806de:::obj544:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:657806de:::obj544:head 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:657806de:::obj544:head v 107'544 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.257+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.257+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:64c1dddc:::obj83:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:64c1dddc:::obj83:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 111/111 lb 1:64c1dddc:::obj83:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:64c1dddc:::obj83:head) v3 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64bc0383:::obj214:head local-lis/les=109/110 n=235 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:64c1dddc:::obj83:head) 2026-02-11T23:31:17.257+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:64c1dddc:::obj83:head) v3 finish latency 0.000011 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.257+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.257+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.257+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa0780 con 0x3752000 2026-02-11T23:31:17.257+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:657806de:::obj544:head)]) -- 0x4f89980 con 0x3752000 2026-02-11T23:31:17.257+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.258+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 511 ==== MOSDPGPush(1.0 111/109 [PushOp(1:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43609c0 con 0x3752000 2026-02-11T23:31:17.258+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.258+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.258+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 512 ==== pg_backfill(progress 1.0 e 111/111 lb 1:657806de:::obj544:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f9e100 con 0x3752000 2026-02-11T23:31:17.258+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:657806de:::obj544:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.258+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:657806de:::obj544:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 111/109 [PushOp(1:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:657806de:::obj544:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:65a1355c:::obj536:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:65a1355c:::obj536:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:65a1355c:::obj536:head 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:65a1355c:::obj536:head v 107'536 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.258+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:657806de:::obj544:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.258+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:657806de:::obj544:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 111/111 lb 1:657806de:::obj544:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.258+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.258+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:657806de:::obj544:head) v3 2026-02-11T23:31:17.258+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:64c1dddc:::obj83:head local-lis/les=109/110 n=236 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:657806de:::obj544:head) 2026-02-11T23:31:17.258+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.258+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:657806de:::obj544:head) v3 finish latency 0.000015 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.258+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.258+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa0900 con 0x3752000 2026-02-11T23:31:17.258+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:65a1355c:::obj536:head)]) -- 0x4f89c80 con 0x3752000 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.258+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 513 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44c2680 con 0x3752000 2026-02-11T23:31:17.258+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.258+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.258+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 514 ==== pg_backfill(progress 1.0 e 111/111 lb 1:65a1355c:::obj536:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f9e680 con 0x3752000 2026-02-11T23:31:17.258+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:65a1355c:::obj536:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 111 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.258+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:65a1355c:::obj536:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 111/109 [PushOp(1:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.258+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:65a1355c:::obj536:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6623e3eb:::obj517:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.258+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6623e3eb:::obj517:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6623e3eb:::obj517:head 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6623e3eb:::obj517:head v 107'517 2026-02-11T23:31:17.258+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.258+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.259+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.259+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:65a1355c:::obj536:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:65a1355c:::obj536:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 111/111 lb 1:65a1355c:::obj536:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:65a1355c:::obj536:head) v3 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:657806de:::obj544:head local-lis/les=109/110 n=237 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:65a1355c:::obj536:head) 2026-02-11T23:31:17.259+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:65a1355c:::obj536:head) v3 finish latency 0.000014 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.259+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.259+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa0a80 con 0x3752000 2026-02-11T23:31:17.259+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6623e3eb:::obj517:head)]) -- 0x4fb8000 con 0x3752000 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.259+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 515 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44c2b60 con 0x3752000 2026-02-11T23:31:17.259+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 111 2026-02-11T23:31:17.259+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.259+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.259+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 516 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6623e3eb:::obj517:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f9ec00 con 0x3752000 2026-02-11T23:31:17.259+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.259+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6623e3eb:::obj517:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 111 2026-02-11T23:31:17.259+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6623e3eb:::obj517:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.259+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.259+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.259+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 111/109 [PushOp(1:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.259+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.259+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6623e3eb:::obj517:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.259+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.259+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6681cfe7:::obj102:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.259+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6681cfe7:::obj102:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.259+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6681cfe7:::obj102:head 2026-02-11T23:31:17.259+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6681cfe7:::obj102:head v 107'102 2026-02-11T23:31:17.259+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.259+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.259+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.259+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6623e3eb:::obj517:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6623e3eb:::obj517:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 1.0 e 111/111 lb 1:6623e3eb:::obj517:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6623e3eb:::obj517:head) v3 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:65a1355c:::obj536:head local-lis/les=109/110 n=238 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6623e3eb:::obj517:head) 2026-02-11T23:31:17.259+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6623e3eb:::obj517:head) v3 finish latency 0.000018 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.259+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.259+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.259+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa0c00 con 0x3752000 2026-02-11T23:31:17.259+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6681cfe7:::obj102:head)]) -- 0x4fb8300 con 0x3752000 2026-02-11T23:31:17.259+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.260+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 517 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x44c3040 con 0x3752000 2026-02-11T23:31:17.260+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 111 2026-02-11T23:31:17.260+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.260+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 518 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6681cfe7:::obj102:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f9f180 con 0x3752000 2026-02-11T23:31:17.260+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6681cfe7:::obj102:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.260+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6681cfe7:::obj102:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 111/109 [PushOp(1:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6681cfe7:::obj102:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6727c4f6:::obj39:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6727c4f6:::obj39:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6727c4f6:::obj39:head 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6727c4f6:::obj39:head v 107'39 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.260+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6681cfe7:::obj102:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.260+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6681cfe7:::obj102:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 111/111 lb 1:6681cfe7:::obj102:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.260+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.260+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6681cfe7:::obj102:head) v3 2026-02-11T23:31:17.260+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6623e3eb:::obj517:head local-lis/les=109/110 n=239 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6681cfe7:::obj102:head) 2026-02-11T23:31:17.260+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.260+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6681cfe7:::obj102:head) v3 finish latency 0.000016 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.260+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.260+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa0d80 con 0x3752000 2026-02-11T23:31:17.260+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6727c4f6:::obj39:head)]) -- 0x4fb8600 con 0x3752000 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.260+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 519 ==== MOSDPGPush(1.0 111/109 [PushOp(1:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x44c3520 con 0x3752000 2026-02-11T23:31:17.260+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 111 2026-02-11T23:31:17.260+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.260+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 520 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6727c4f6:::obj39:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f9f700 con 0x3752000 2026-02-11T23:31:17.260+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6727c4f6:::obj39:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 111 2026-02-11T23:31:17.260+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6727c4f6:::obj39:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 111/109 [PushOp(1:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6727c4f6:::obj39:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:67a26c0a:::obj53:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:67a26c0a:::obj53:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:67a26c0a:::obj53:head 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:67a26c0a:::obj53:head v 107'53 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.260+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.260+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.260+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6727c4f6:::obj39:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.260+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6727c4f6:::obj39:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 111/111 lb 1:6727c4f6:::obj39:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.260+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.260+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6727c4f6:::obj39:head) v3 2026-02-11T23:31:17.260+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6681cfe7:::obj102:head local-lis/les=109/110 n=240 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6727c4f6:::obj39:head) 2026-02-11T23:31:17.260+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.260+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6727c4f6:::obj39:head) v3 finish latency 0.000014 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.260+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.261+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.261+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.261+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa0f00 con 0x3752000 2026-02-11T23:31:17.261+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:67a26c0a:::obj53:head)]) -- 0x4fb8780 con 0x3752000 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.261+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 521 ==== MOSDPGPush(1.0 111/109 [PushOp(1:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44c3a00 con 0x3752000 2026-02-11T23:31:17.261+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 111 2026-02-11T23:31:17.261+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.261+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.261+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 522 ==== pg_backfill(progress 1.0 e 111/111 lb 1:67a26c0a:::obj53:head) ==== 1061+0+0 (unknown 0 0 0) 0x4fca000 con 0x3752000 2026-02-11T23:31:17.261+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.261+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:67a26c0a:::obj53:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.261+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:67a26c0a:::obj53:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.261+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.261+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.261+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 111/109 [PushOp(1:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.261+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.261+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.261+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:67a26c0a:::obj53:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.261+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:685e1da9:::obj264:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.261+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:685e1da9:::obj264:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.261+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:685e1da9:::obj264:head 2026-02-11T23:31:17.261+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:685e1da9:::obj264:head v 107'264 2026-02-11T23:31:17.261+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.261+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.261+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.261+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:67a26c0a:::obj53:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.261+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:67a26c0a:::obj53:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 111/111 lb 1:67a26c0a:::obj53:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.261+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.261+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:67a26c0a:::obj53:head) v3 2026-02-11T23:31:17.261+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6727c4f6:::obj39:head local-lis/les=109/110 n=241 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:67a26c0a:::obj53:head) 2026-02-11T23:31:17.261+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.261+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:67a26c0a:::obj53:head) v3 finish latency 0.000019 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.261+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.261+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.261+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa1080 con 0x3752000 2026-02-11T23:31:17.261+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:685e1da9:::obj264:head)]) -- 0x4fb8900 con 0x3752000 2026-02-11T23:31:17.261+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.262+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 523 ==== MOSDPGPush(1.0 111/109 [PushOp(1:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4360000 con 0x3752000 2026-02-11T23:31:17.262+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.262+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.262+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 524 ==== pg_backfill(progress 1.0 e 111/111 lb 1:685e1da9:::obj264:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fca580 con 0x3752000 2026-02-11T23:31:17.262+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:685e1da9:::obj264:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.262+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:685e1da9:::obj264:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 111/109 [PushOp(1:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:685e1da9:::obj264:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:685e6e29:::obj176:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:685e6e29:::obj176:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:685e6e29:::obj176:head 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:685e6e29:::obj176:head v 107'176 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.262+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:685e1da9:::obj264:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.262+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:685e1da9:::obj264:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 111/111 lb 1:685e1da9:::obj264:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.262+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.262+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:685e1da9:::obj264:head) v3 2026-02-11T23:31:17.262+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:67a26c0a:::obj53:head local-lis/les=109/110 n=242 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:685e1da9:::obj264:head) 2026-02-11T23:31:17.262+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.262+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:685e1da9:::obj264:head) v3 finish latency 0.000016 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.262+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.262+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa1200 con 0x3752000 2026-02-11T23:31:17.262+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:685e6e29:::obj176:head)]) -- 0x4fb8c00 con 0x3752000 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.262+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 525 ==== MOSDPGPush(1.0 111/109 [PushOp(1:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44bbd40 con 0x3752000 2026-02-11T23:31:17.262+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.262+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.262+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 526 ==== pg_backfill(progress 1.0 e 111/111 lb 1:685e6e29:::obj176:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fcab00 con 0x3752000 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.262+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:685e6e29:::obj176:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 111 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.262+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:685e6e29:::obj176:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 111/109 [PushOp(1:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:685e6e29:::obj176:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:68743261:::obj370:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:68743261:::obj370:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:68743261:::obj370:head 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:68743261:::obj370:head v 107'370 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.262+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.262+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.262+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:685e6e29:::obj176:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.262+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:685e6e29:::obj176:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 111/111 lb 1:685e6e29:::obj176:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.262+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.262+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:685e6e29:::obj176:head) v3 2026-02-11T23:31:17.262+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e1da9:::obj264:head local-lis/les=109/110 n=243 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:685e6e29:::obj176:head) 2026-02-11T23:31:17.262+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.262+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:685e6e29:::obj176:head) v3 finish latency 0.000014 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.262+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.263+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.263+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.263+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa1380 con 0x3752000 2026-02-11T23:31:17.263+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:68743261:::obj370:head)]) -- 0x4fb8f00 con 0x3752000 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.263+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 527 ==== MOSDPGPush(1.0 111/109 [PushOp(1:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44c2340 con 0x3752000 2026-02-11T23:31:17.263+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 111 2026-02-11T23:31:17.263+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.263+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.263+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 528 ==== pg_backfill(progress 1.0 e 111/111 lb 1:68743261:::obj370:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fcb080 con 0x3752000 2026-02-11T23:31:17.263+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:68743261:::obj370:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 111 2026-02-11T23:31:17.263+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.263+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:68743261:::obj370:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.263+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.263+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.263+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 111/109 [PushOp(1:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.263+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:68743261:::obj370:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.263+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.263+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.263+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:68869f42:::obj433:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.263+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:68869f42:::obj433:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.263+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:68869f42:::obj433:head 2026-02-11T23:31:17.263+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:68869f42:::obj433:head v 107'433 2026-02-11T23:31:17.263+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.263+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.263+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:17.263+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:68743261:::obj370:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.263+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:68743261:::obj370:head) v3 prio 15 cost 0 latency 0.000101 pg_backfill(progress 1.0 e 111/111 lb 1:68743261:::obj370:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.263+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.263+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:68743261:::obj370:head) v3 2026-02-11T23:31:17.263+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:685e6e29:::obj176:head local-lis/les=109/110 n=244 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:68743261:::obj370:head) 2026-02-11T23:31:17.263+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.263+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:68743261:::obj370:head) v3 finish latency 0.000012 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.263+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.263+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.263+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa1500 con 0x3752000 2026-02-11T23:31:17.263+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:68869f42:::obj433:head)]) -- 0x4fb9200 con 0x3752000 2026-02-11T23:31:17.263+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.264+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 529 ==== MOSDPGPush(1.0 111/109 [PushOp(1:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44bb040 con 0x3752000 2026-02-11T23:31:17.264+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 111 2026-02-11T23:31:17.264+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.264+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 530 ==== pg_backfill(progress 1.0 e 111/111 lb 1:68869f42:::obj433:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fcb600 con 0x3752000 2026-02-11T23:31:17.264+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:68869f42:::obj433:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.264+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:68869f42:::obj433:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 111/109 [PushOp(1:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:68869f42:::obj433:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:69182b79:::obj301:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:69182b79:::obj301:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:69182b79:::obj301:head 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:69182b79:::obj301:head v 107'301 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.264+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:68869f42:::obj433:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:68869f42:::obj433:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 1.0 e 111/111 lb 1:68869f42:::obj433:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:68869f42:::obj433:head) v3 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68743261:::obj370:head local-lis/les=109/110 n=245 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:68869f42:::obj433:head) 2026-02-11T23:31:17.264+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:68869f42:::obj433:head) v3 finish latency 0.000016 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.264+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa1680 con 0x3752000 2026-02-11T23:31:17.264+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:69182b79:::obj301:head)]) -- 0x4fb9500 con 0x3752000 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.264+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 531 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44bb520 con 0x3752000 2026-02-11T23:31:17.264+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.264+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.264+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 532 ==== pg_backfill(progress 1.0 e 111/111 lb 1:69182b79:::obj301:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fcbb80 con 0x3752000 2026-02-11T23:31:17.264+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:69182b79:::obj301:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.264+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:69182b79:::obj301:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 111/109 [PushOp(1:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:69182b79:::obj301:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6934f76e:::obj318:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6934f76e:::obj318:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6934f76e:::obj318:head 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6934f76e:::obj318:head v 107'318 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.264+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.264+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.264+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:69182b79:::obj301:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:69182b79:::obj301:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 111/111 lb 1:69182b79:::obj301:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:69182b79:::obj301:head) v3 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:68869f42:::obj433:head local-lis/les=109/110 n=246 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:69182b79:::obj301:head) 2026-02-11T23:31:17.264+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.264+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:69182b79:::obj301:head) v3 finish latency 0.000017 2026-02-11T23:31:17.264+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.265+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.265+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.265+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa1b00 con 0x3752000 2026-02-11T23:31:17.265+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6934f76e:::obj318:head)]) -- 0x4fa1800 con 0x3752000 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.265+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 533 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43e5ba0 con 0x3752000 2026-02-11T23:31:17.265+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 111 2026-02-11T23:31:17.265+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.265+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.265+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 534 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6934f76e:::obj318:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fcc100 con 0x3752000 2026-02-11T23:31:17.265+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.265+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6934f76e:::obj318:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.265+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6934f76e:::obj318:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.265+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.265+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.265+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 111/109 [PushOp(1:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.265+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6934f76e:::obj318:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.265+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.265+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.265+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6971da6a:::obj293:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.265+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6971da6a:::obj293:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.265+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6971da6a:::obj293:head 2026-02-11T23:31:17.265+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6971da6a:::obj293:head v 107'293 2026-02-11T23:31:17.265+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.265+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.265+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.265+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6934f76e:::obj318:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.265+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6934f76e:::obj318:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 111/111 lb 1:6934f76e:::obj318:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.265+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.265+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6934f76e:::obj318:head) v3 2026-02-11T23:31:17.265+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69182b79:::obj301:head local-lis/les=109/110 n=247 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6934f76e:::obj318:head) 2026-02-11T23:31:17.265+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.265+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6934f76e:::obj318:head) v3 finish latency 0.000014 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.265+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.265+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.265+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa1c80 con 0x3752000 2026-02-11T23:31:17.265+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6971da6a:::obj293:head)]) -- 0x4fb9800 con 0x3752000 2026-02-11T23:31:17.265+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.266+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 535 ==== MOSDPGPush(1.0 111/109 [PushOp(1:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3944340 con 0x3752000 2026-02-11T23:31:17.266+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 111 2026-02-11T23:31:17.266+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.266+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 536 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6971da6a:::obj293:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fcc680 con 0x3752000 2026-02-11T23:31:17.266+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6971da6a:::obj293:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.266+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6971da6a:::obj293:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 111/109 [PushOp(1:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6971da6a:::obj293:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:698e6c1e:::obj207:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:698e6c1e:::obj207:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:698e6c1e:::obj207:head 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:698e6c1e:::obj207:head v 107'207 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.266+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6971da6a:::obj293:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6971da6a:::obj293:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 111/111 lb 1:6971da6a:::obj293:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6971da6a:::obj293:head) v3 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6934f76e:::obj318:head local-lis/les=109/110 n=248 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6971da6a:::obj293:head) 2026-02-11T23:31:17.266+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6971da6a:::obj293:head) v3 finish latency 0.000013 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.266+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa1e00 con 0x3752000 2026-02-11T23:31:17.266+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:698e6c1e:::obj207:head)]) -- 0x4fb9b00 con 0x3752000 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.266+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 537 ==== MOSDPGPush(1.0 111/109 [PushOp(1:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x3944820 con 0x3752000 2026-02-11T23:31:17.266+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 111 2026-02-11T23:31:17.266+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.266+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 538 ==== pg_backfill(progress 1.0 e 111/111 lb 1:698e6c1e:::obj207:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fccc00 con 0x3752000 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.266+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:698e6c1e:::obj207:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 111 2026-02-11T23:31:17.266+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:698e6c1e:::obj207:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 111/109 [PushOp(1:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:698e6c1e:::obj207:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:699d4764:::obj50:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:699d4764:::obj50:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:699d4764:::obj50:head 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:699d4764:::obj50:head v 107'50 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.266+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:698e6c1e:::obj207:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:698e6c1e:::obj207:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 111/111 lb 1:698e6c1e:::obj207:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:698e6c1e:::obj207:head) v3 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6971da6a:::obj293:head local-lis/les=109/110 n=249 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:698e6c1e:::obj207:head) 2026-02-11T23:31:17.266+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.266+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:698e6c1e:::obj207:head) v3 finish latency 0.000011 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.266+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.266+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.266+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4180 con 0x3752000 2026-02-11T23:31:17.266+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:699d4764:::obj50:head)]) -- 0x4fe4000 con 0x3752000 2026-02-11T23:31:17.266+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.267+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 539 ==== MOSDPGPush(1.0 111/109 [PushOp(1:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x46051e0 con 0x3752000 2026-02-11T23:31:17.267+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 111 2026-02-11T23:31:17.267+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.267+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 540 ==== pg_backfill(progress 1.0 e 111/111 lb 1:699d4764:::obj50:head) ==== 1061+0+0 (unknown 0 0 0) 0x4fcd180 con 0x3752000 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.267+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:699d4764:::obj50:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 111 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.267+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:699d4764:::obj50:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 111/109 [PushOp(1:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:699d4764:::obj50:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:69f21cc4:::obj383:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:69f21cc4:::obj383:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:69f21cc4:::obj383:head 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:69f21cc4:::obj383:head v 107'383 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.267+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:699d4764:::obj50:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.267+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:699d4764:::obj50:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 111/111 lb 1:699d4764:::obj50:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.267+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.267+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:699d4764:::obj50:head) v3 2026-02-11T23:31:17.267+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:698e6c1e:::obj207:head local-lis/les=109/110 n=250 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:699d4764:::obj50:head) 2026-02-11T23:31:17.267+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.267+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:699d4764:::obj50:head) v3 finish latency 0.000017 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.267+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.267+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4300 con 0x3752000 2026-02-11T23:31:17.267+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:69f21cc4:::obj383:head)]) -- 0x4fb9e00 con 0x3752000 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.267+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 541 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x46056c0 con 0x3752000 2026-02-11T23:31:17.267+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 111 2026-02-11T23:31:17.267+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.267+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 542 ==== pg_backfill(progress 1.0 e 111/111 lb 1:69f21cc4:::obj383:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fcd700 con 0x3752000 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.267+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:69f21cc4:::obj383:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 111 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.267+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:69f21cc4:::obj383:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 111/109 [PushOp(1:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:69f21cc4:::obj383:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6c7994f7:::obj432:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6c7994f7:::obj432:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6c7994f7:::obj432:head 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6c7994f7:::obj432:head v 107'432 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.267+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.267+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:17.267+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:69f21cc4:::obj383:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.267+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:69f21cc4:::obj383:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 111/111 lb 1:69f21cc4:::obj383:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.267+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.267+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:69f21cc4:::obj383:head) v3 2026-02-11T23:31:17.267+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:699d4764:::obj50:head local-lis/les=109/110 n=251 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:69f21cc4:::obj383:head) 2026-02-11T23:31:17.267+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.267+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:69f21cc4:::obj383:head) v3 finish latency 0.000016 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.267+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.268+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.268+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.268+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4480 con 0x3752000 2026-02-11T23:31:17.268+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6c7994f7:::obj432:head)]) -- 0x4fea180 con 0x3752000 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.268+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 543 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4603040 con 0x3752000 2026-02-11T23:31:17.268+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.268+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.268+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.268+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.268+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 544 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6c7994f7:::obj432:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fec000 con 0x3752000 2026-02-11T23:31:17.268+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.268+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.268+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6c7994f7:::obj432:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 111 2026-02-11T23:31:17.268+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6c7994f7:::obj432:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.268+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 111/109 [PushOp(1:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.268+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.268+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.268+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6c7994f7:::obj432:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.268+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6dc7b1fd:::obj49:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.268+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6dc7b1fd:::obj49:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.268+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6dc7b1fd:::obj49:head 2026-02-11T23:31:17.268+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6dc7b1fd:::obj49:head v 107'49 2026-02-11T23:31:17.268+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.268+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.268+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.268+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6c7994f7:::obj432:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.268+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6c7994f7:::obj432:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 111/111 lb 1:6c7994f7:::obj432:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.268+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.268+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6c7994f7:::obj432:head) v3 2026-02-11T23:31:17.268+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:69f21cc4:::obj383:head local-lis/les=109/110 n=252 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6c7994f7:::obj432:head) 2026-02-11T23:31:17.268+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.268+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6c7994f7:::obj432:head) v3 finish latency 0.000019 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.268+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.268+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.268+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4600 con 0x3752000 2026-02-11T23:31:17.268+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6dc7b1fd:::obj49:head)]) -- 0x4fea480 con 0x3752000 2026-02-11T23:31:17.268+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.269+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 545 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4603520 con 0x3752000 2026-02-11T23:31:17.269+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 111 2026-02-11T23:31:17.269+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.269+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 546 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6dc7b1fd:::obj49:head) ==== 1061+0+0 (unknown 0 0 0) 0x4fec580 con 0x3752000 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.269+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6dc7b1fd:::obj49:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 111 2026-02-11T23:31:17.269+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6dc7b1fd:::obj49:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 111/109 [PushOp(1:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6dc7b1fd:::obj49:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6e0bfbf2:::obj372:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6e0bfbf2:::obj372:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6e0bfbf2:::obj372:head 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6e0bfbf2:::obj372:head v 107'372 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.269+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6dc7b1fd:::obj49:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6dc7b1fd:::obj49:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 111/111 lb 1:6dc7b1fd:::obj49:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6dc7b1fd:::obj49:head) v3 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6c7994f7:::obj432:head local-lis/les=109/110 n=253 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6dc7b1fd:::obj49:head) 2026-02-11T23:31:17.269+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6dc7b1fd:::obj49:head) v3 finish latency 0.000015 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.269+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4780 con 0x3752000 2026-02-11T23:31:17.269+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6e0bfbf2:::obj372:head)]) -- 0x4fea600 con 0x3752000 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.269+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 547 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4603a00 con 0x3752000 2026-02-11T23:31:17.269+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 111 2026-02-11T23:31:17.269+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.269+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 548 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6e0bfbf2:::obj372:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fecb00 con 0x3752000 2026-02-11T23:31:17.269+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6e0bfbf2:::obj372:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 111 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.269+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6e0bfbf2:::obj372:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 111/109 [PushOp(1:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6e0bfbf2:::obj372:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6ec34796:::obj408:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6ec34796:::obj408:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6ec34796:::obj408:head 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6ec34796:::obj408:head v 107'408 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.269+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.269+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6e0bfbf2:::obj372:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6e0bfbf2:::obj372:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 111/111 lb 1:6e0bfbf2:::obj372:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6e0bfbf2:::obj372:head) v3 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6dc7b1fd:::obj49:head local-lis/les=109/110 n=254 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6e0bfbf2:::obj372:head) 2026-02-11T23:31:17.269+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6e0bfbf2:::obj372:head) v3 finish latency 0.000015 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.269+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.269+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.269+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4900 con 0x3752000 2026-02-11T23:31:17.269+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6ec34796:::obj408:head)]) -- 0x4fea900 con 0x3752000 2026-02-11T23:31:17.269+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.270+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 549 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3944d00 con 0x3752000 2026-02-11T23:31:17.270+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 111 2026-02-11T23:31:17.270+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.270+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 550 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6ec34796:::obj408:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fed080 con 0x3752000 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.270+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6ec34796:::obj408:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 111 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.270+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6ec34796:::obj408:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 111/109 [PushOp(1:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6ec34796:::obj408:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6ec93c40:::obj366:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6ec93c40:::obj366:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6ec93c40:::obj366:head 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6ec93c40:::obj366:head v 107'366 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.270+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6ec34796:::obj408:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.270+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6ec34796:::obj408:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 111/111 lb 1:6ec34796:::obj408:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.270+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.270+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6ec34796:::obj408:head) v3 2026-02-11T23:31:17.270+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6e0bfbf2:::obj372:head local-lis/les=109/110 n=255 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6ec34796:::obj408:head) 2026-02-11T23:31:17.270+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.270+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6ec34796:::obj408:head) v3 finish latency 0.000015 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.270+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.270+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4a80 con 0x3752000 2026-02-11T23:31:17.270+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6ec93c40:::obj366:head)]) -- 0x4feac00 con 0x3752000 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.270+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 551 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39451e0 con 0x3752000 2026-02-11T23:31:17.270+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 111 2026-02-11T23:31:17.270+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.270+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 552 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6ec93c40:::obj366:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fed600 con 0x3752000 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.270+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6ec93c40:::obj366:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.270+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6ec93c40:::obj366:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 111/109 [PushOp(1:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.270+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6ec93c40:::obj366:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6ec9e2eb:::obj591:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.270+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6ec9e2eb:::obj591:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6ec9e2eb:::obj591:head 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6ec9e2eb:::obj591:head v 107'591 2026-02-11T23:31:17.270+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.270+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.271+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.271+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6ec93c40:::obj366:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6ec93c40:::obj366:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 111/111 lb 1:6ec93c40:::obj366:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6ec93c40:::obj366:head) v3 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec34796:::obj408:head local-lis/les=109/110 n=256 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6ec93c40:::obj366:head) 2026-02-11T23:31:17.271+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6ec93c40:::obj366:head) v3 finish latency 0.000015 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.271+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.271+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4c00 con 0x3752000 2026-02-11T23:31:17.271+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6ec9e2eb:::obj591:head)]) -- 0x4feaf00 con 0x3752000 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.271+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 553 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39456c0 con 0x3752000 2026-02-11T23:31:17.271+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.271+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.271+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.271+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 554 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6ec9e2eb:::obj591:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fedb80 con 0x3752000 2026-02-11T23:31:17.271+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.271+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6ec9e2eb:::obj591:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 111 2026-02-11T23:31:17.271+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6ec9e2eb:::obj591:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.271+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.271+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.271+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 111/109 [PushOp(1:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.271+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.271+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.271+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6ec9e2eb:::obj591:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.271+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6f4176ae:::obj571:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.271+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6f4176ae:::obj571:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.271+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6f4176ae:::obj571:head 2026-02-11T23:31:17.271+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6f4176ae:::obj571:head v 107'571 2026-02-11T23:31:17.271+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.271+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.271+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.271+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6ec9e2eb:::obj591:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6ec9e2eb:::obj591:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 111/111 lb 1:6ec9e2eb:::obj591:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6ec9e2eb:::obj591:head) v3 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec93c40:::obj366:head local-lis/les=109/110 n=257 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6ec9e2eb:::obj591:head) 2026-02-11T23:31:17.271+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6ec9e2eb:::obj591:head) v3 finish latency 0.000015 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.271+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.271+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.271+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4d80 con 0x3752000 2026-02-11T23:31:17.271+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6f4176ae:::obj571:head)]) -- 0x4feb200 con 0x3752000 2026-02-11T23:31:17.271+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.272+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 555 ==== MOSDPGPush(1.0 111/109 [PushOp(1:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x3945ba0 con 0x3752000 2026-02-11T23:31:17.272+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 111 2026-02-11T23:31:17.272+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.272+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 556 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6f4176ae:::obj571:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fee100 con 0x3752000 2026-02-11T23:31:17.272+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6f4176ae:::obj571:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 111 2026-02-11T23:31:17.272+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6f4176ae:::obj571:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 111/109 [PushOp(1:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6f4176ae:::obj571:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:6f5862a4:::obj430:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:6f5862a4:::obj430:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:6f5862a4:::obj430:head 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:6f5862a4:::obj430:head v 107'430 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.272+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6f4176ae:::obj571:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6f4176ae:::obj571:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 111/111 lb 1:6f4176ae:::obj571:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6f4176ae:::obj571:head) v3 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6ec9e2eb:::obj591:head local-lis/les=109/110 n=258 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6f4176ae:::obj571:head) 2026-02-11T23:31:17.272+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6f4176ae:::obj571:head) v3 finish latency 0.000012 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.272+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe5200 con 0x3752000 2026-02-11T23:31:17.272+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:6f5862a4:::obj430:head)]) -- 0x4fe4f00 con 0x3752000 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.272+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 557 ==== MOSDPGPush(1.0 111/109 [PushOp(1:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39441a0 con 0x3752000 2026-02-11T23:31:17.272+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.272+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.272+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 558 ==== pg_backfill(progress 1.0 e 111/111 lb 1:6f5862a4:::obj430:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fee680 con 0x3752000 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.272+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:6f5862a4:::obj430:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 111 2026-02-11T23:31:17.272+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6f5862a4:::obj430:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 111/109 [PushOp(1:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6f5862a4:::obj430:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:70360d67:::obj288:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:70360d67:::obj288:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:70360d67:::obj288:head 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:70360d67:::obj288:head v 107'288 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.272+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.272+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:6f5862a4:::obj430:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6f5862a4:::obj430:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 111/111 lb 1:6f5862a4:::obj430:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:6f5862a4:::obj430:head) v3 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f4176ae:::obj571:head local-lis/les=109/110 n=259 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:6f5862a4:::obj430:head) 2026-02-11T23:31:17.272+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:6f5862a4:::obj430:head) v3 finish latency 0.000014 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.272+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.272+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.272+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe5380 con 0x3752000 2026-02-11T23:31:17.272+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:70360d67:::obj288:head)]) -- 0x4feb500 con 0x3752000 2026-02-11T23:31:17.272+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.273+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 559 ==== MOSDPGPush(1.0 111/109 [PushOp(1:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4605ba0 con 0x3752000 2026-02-11T23:31:17.273+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.273+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.273+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 560 ==== pg_backfill(progress 1.0 e 111/111 lb 1:70360d67:::obj288:head) ==== 1062+0+0 (unknown 0 0 0) 0x4feec00 con 0x3752000 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.273+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:70360d67:::obj288:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.273+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:70360d67:::obj288:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:70360d67:::obj288:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 111/109 [PushOp(1:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:707a62d6:::obj465:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:707a62d6:::obj465:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:707a62d6:::obj465:head 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:707a62d6:::obj465:head v 107'465 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.273+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:70360d67:::obj288:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.273+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:70360d67:::obj288:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 111/111 lb 1:70360d67:::obj288:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.273+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.273+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:70360d67:::obj288:head) v3 2026-02-11T23:31:17.273+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:6f5862a4:::obj430:head local-lis/les=109/110 n=260 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:70360d67:::obj288:head) 2026-02-11T23:31:17.273+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.273+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:70360d67:::obj288:head) v3 finish latency 0.000015 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.273+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.273+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe5500 con 0x3752000 2026-02-11T23:31:17.273+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:707a62d6:::obj465:head)]) -- 0x4feb800 con 0x3752000 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.273+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 561 ==== MOSDPGPush(1.0 111/109 [PushOp(1:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x38f4d00 con 0x3752000 2026-02-11T23:31:17.273+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.273+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.273+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 562 ==== pg_backfill(progress 1.0 e 111/111 lb 1:707a62d6:::obj465:head) ==== 1062+0+0 (unknown 0 0 0) 0x4fef180 con 0x3752000 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.273+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:707a62d6:::obj465:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.273+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:707a62d6:::obj465:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 111/109 [PushOp(1:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.273+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:707a62d6:::obj465:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7109478f:::obj25:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.273+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7109478f:::obj25:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7109478f:::obj25:head 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7109478f:::obj25:head v 107'25 2026-02-11T23:31:17.273+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.273+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.274+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.274+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:707a62d6:::obj465:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:707a62d6:::obj465:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 111/111 lb 1:707a62d6:::obj465:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:707a62d6:::obj465:head) v3 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:70360d67:::obj288:head local-lis/les=109/110 n=261 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:707a62d6:::obj465:head) 2026-02-11T23:31:17.274+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:707a62d6:::obj465:head) v3 finish latency 0.000015 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.274+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.274+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe5680 con 0x3752000 2026-02-11T23:31:17.274+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:7109478f:::obj25:head)]) -- 0x4febb00 con 0x3752000 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.274+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 563 ==== MOSDPGPush(1.0 111/109 [PushOp(1:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x38f51e0 con 0x3752000 2026-02-11T23:31:17.274+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.274+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.274+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 564 ==== pg_backfill(progress 1.0 e 111/111 lb 1:7109478f:::obj25:head) ==== 1061+0+0 (unknown 0 0 0) 0x4fef700 con 0x3752000 2026-02-11T23:31:17.274+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:7109478f:::obj25:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.274+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7109478f:::obj25:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 111/109 [PushOp(1:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.274+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.274+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7109478f:::obj25:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7135fd38:::obj57:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7135fd38:::obj57:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7135fd38:::obj57:head 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7135fd38:::obj57:head v 107'57 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.274+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.274+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.274+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7109478f:::obj25:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.274+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7109478f:::obj25:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 111/111 lb 1:7109478f:::obj25:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.274+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.274+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:7109478f:::obj25:head) v3 2026-02-11T23:31:17.274+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:707a62d6:::obj465:head local-lis/les=109/110 n=262 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:7109478f:::obj25:head) 2026-02-11T23:31:17.274+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.274+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7109478f:::obj25:head) v3 finish latency 0.000012 2026-02-11T23:31:17.274+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.274+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.274+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.274+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe5980 con 0x3752000 2026-02-11T23:31:17.274+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:7135fd38:::obj57:head)]) -- 0x4fe5800 con 0x3752000 2026-02-11T23:31:17.274+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.275+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 565 ==== MOSDPGPush(1.0 111/109 [PushOp(1:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x38f56c0 con 0x3752000 2026-02-11T23:31:17.275+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.275+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.275+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 566 ==== pg_backfill(progress 1.0 e 111/111 lb 1:7135fd38:::obj57:head) ==== 1061+0+0 (unknown 0 0 0) 0x4d92000 con 0x3752000 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.275+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:7135fd38:::obj57:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 111 2026-02-11T23:31:17.275+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7135fd38:::obj57:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 111/109 [PushOp(1:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7135fd38:::obj57:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:72da0e1a:::obj147:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:72da0e1a:::obj147:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:72da0e1a:::obj147:head 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:72da0e1a:::obj147:head v 107'147 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.275+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7135fd38:::obj57:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7135fd38:::obj57:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 111/111 lb 1:7135fd38:::obj57:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:7135fd38:::obj57:head) v3 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7109478f:::obj25:head local-lis/les=109/110 n=263 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:7135fd38:::obj57:head) 2026-02-11T23:31:17.275+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7135fd38:::obj57:head) v3 finish latency 0.000014 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.275+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe5b00 con 0x3752000 2026-02-11T23:31:17.275+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:72da0e1a:::obj147:head)]) -- 0x4febc80 con 0x3752000 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.275+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 567 ==== MOSDPGPush(1.0 111/109 [PushOp(1:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x38f41a0 con 0x3752000 2026-02-11T23:31:17.275+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.275+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.275+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 568 ==== pg_backfill(progress 1.0 e 111/111 lb 1:72da0e1a:::obj147:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d92580 con 0x3752000 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.275+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:72da0e1a:::obj147:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 111 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.275+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:72da0e1a:::obj147:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 111/109 [PushOp(1:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:72da0e1a:::obj147:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:73627f05:::obj80:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:73627f05:::obj80:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:73627f05:::obj80:head 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:73627f05:::obj80:head v 107'80 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.275+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.275+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:72da0e1a:::obj147:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:72da0e1a:::obj147:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 111/111 lb 1:72da0e1a:::obj147:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:72da0e1a:::obj147:head) v3 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7135fd38:::obj57:head local-lis/les=109/110 n=264 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:72da0e1a:::obj147:head) 2026-02-11T23:31:17.275+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:72da0e1a:::obj147:head) v3 finish latency 0.000015 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.275+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.275+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.275+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe5c80 con 0x3752000 2026-02-11T23:31:17.275+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:73627f05:::obj80:head)]) -- 0x4d96000 con 0x3752000 2026-02-11T23:31:17.275+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.276+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 569 ==== MOSDPGPush(1.0 111/109 [PushOp(1:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44a6d00 con 0x3752000 2026-02-11T23:31:17.276+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.276+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.276+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 570 ==== pg_backfill(progress 1.0 e 111/111 lb 1:73627f05:::obj80:head) ==== 1061+0+0 (unknown 0 0 0) 0x4d92b00 con 0x3752000 2026-02-11T23:31:17.276+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:73627f05:::obj80:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.276+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:73627f05:::obj80:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 111/109 [PushOp(1:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:73627f05:::obj80:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:737833f6:::obj391:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:737833f6:::obj391:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:737833f6:::obj391:head 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:737833f6:::obj391:head v 107'391 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.276+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:73627f05:::obj80:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.276+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:73627f05:::obj80:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 111/111 lb 1:73627f05:::obj80:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.276+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.276+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:73627f05:::obj80:head) v3 2026-02-11T23:31:17.276+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:72da0e1a:::obj147:head local-lis/les=109/110 n=265 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:73627f05:::obj80:head) 2026-02-11T23:31:17.276+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.276+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:73627f05:::obj80:head) v3 finish latency 0.000015 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.276+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.276+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe5e00 con 0x3752000 2026-02-11T23:31:17.276+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:737833f6:::obj391:head)]) -- 0x4d96180 con 0x3752000 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.276+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 571 ==== MOSDPGPush(1.0 111/109 [PushOp(1:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44a71e0 con 0x3752000 2026-02-11T23:31:17.276+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.276+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.276+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 572 ==== pg_backfill(progress 1.0 e 111/111 lb 1:737833f6:::obj391:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d93080 con 0x3752000 2026-02-11T23:31:17.276+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:737833f6:::obj391:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.276+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:737833f6:::obj391:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 111/109 [PushOp(1:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:737833f6:::obj391:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:744a8d59:::obj243:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:744a8d59:::obj243:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:744a8d59:::obj243:head 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:744a8d59:::obj243:head v 107'243 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.276+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.276+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.276+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:737833f6:::obj391:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.276+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:737833f6:::obj391:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 111/111 lb 1:737833f6:::obj391:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.276+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.276+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:737833f6:::obj391:head) v3 2026-02-11T23:31:17.276+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:73627f05:::obj80:head local-lis/les=109/110 n=266 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:737833f6:::obj391:head) 2026-02-11T23:31:17.276+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.276+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:737833f6:::obj391:head) v3 finish latency 0.000015 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.276+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.277+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.277+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.277+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4d80 con 0x3752000 2026-02-11T23:31:17.277+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:744a8d59:::obj243:head)]) -- 0x4d96480 con 0x3752000 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.277+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 573 ==== MOSDPGPush(1.0 111/109 [PushOp(1:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44a76c0 con 0x3752000 2026-02-11T23:31:17.277+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.277+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.277+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 574 ==== pg_backfill(progress 1.0 e 111/111 lb 1:744a8d59:::obj243:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d93600 con 0x3752000 2026-02-11T23:31:17.277+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:744a8d59:::obj243:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.277+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.277+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:744a8d59:::obj243:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.277+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.277+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.277+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.277+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 111/109 [PushOp(1:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.277+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.277+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:744a8d59:::obj243:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.277+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.277+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:748dbcc6:::obj428:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.277+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:748dbcc6:::obj428:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.277+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:748dbcc6:::obj428:head 2026-02-11T23:31:17.277+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:748dbcc6:::obj428:head v 107'428 2026-02-11T23:31:17.277+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.277+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.277+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.277+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:744a8d59:::obj243:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.277+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:744a8d59:::obj243:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 111/111 lb 1:744a8d59:::obj243:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.277+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.277+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:744a8d59:::obj243:head) v3 2026-02-11T23:31:17.277+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:737833f6:::obj391:head local-lis/les=109/110 n=267 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:744a8d59:::obj243:head) 2026-02-11T23:31:17.277+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.277+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:744a8d59:::obj243:head) v3 finish latency 0.000015 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.277+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.277+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.277+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4feaf00 con 0x3752000 2026-02-11T23:31:17.277+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:748dbcc6:::obj428:head)]) -- 0x4d96780 con 0x3752000 2026-02-11T23:31:17.277+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.278+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 575 ==== MOSDPGPush(1.0 111/109 [PushOp(1:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x44a7ba0 con 0x3752000 2026-02-11T23:31:17.278+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.278+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.278+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 576 ==== pg_backfill(progress 1.0 e 111/111 lb 1:748dbcc6:::obj428:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d93b80 con 0x3752000 2026-02-11T23:31:17.278+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:748dbcc6:::obj428:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 111 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.278+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:748dbcc6:::obj428:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 111/109 [PushOp(1:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:748dbcc6:::obj428:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7492d19a:::obj168:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7492d19a:::obj168:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7492d19a:::obj168:head 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7492d19a:::obj168:head v 107'168 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.278+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:748dbcc6:::obj428:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:748dbcc6:::obj428:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 111/111 lb 1:748dbcc6:::obj428:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:748dbcc6:::obj428:head) v3 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:744a8d59:::obj243:head local-lis/les=109/110 n=268 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:748dbcc6:::obj428:head) 2026-02-11T23:31:17.278+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:748dbcc6:::obj428:head) v3 finish latency 0.000014 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.278+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4c00 con 0x3752000 2026-02-11T23:31:17.278+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:7492d19a:::obj168:head)]) -- 0x4d96a80 con 0x3752000 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.278+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 577 ==== MOSDPGPush(1.0 111/109 [PushOp(1:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389a000 con 0x3752000 2026-02-11T23:31:17.278+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.278+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.278+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 578 ==== pg_backfill(progress 1.0 e 111/111 lb 1:7492d19a:::obj168:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d94100 con 0x3752000 2026-02-11T23:31:17.278+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:7492d19a:::obj168:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 111 2026-02-11T23:31:17.278+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7492d19a:::obj168:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 111/109 [PushOp(1:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7492d19a:::obj168:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:750d67aa:::obj572:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:750d67aa:::obj572:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:750d67aa:::obj572:head 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:750d67aa:::obj572:head v 107'572 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.278+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.278+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7492d19a:::obj168:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7492d19a:::obj168:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 111/111 lb 1:7492d19a:::obj168:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:7492d19a:::obj168:head) v3 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:748dbcc6:::obj428:head local-lis/les=109/110 n=269 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:7492d19a:::obj168:head) 2026-02-11T23:31:17.278+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7492d19a:::obj168:head) v3 finish latency 0.000012 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.278+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.278+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.278+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4feac00 con 0x3752000 2026-02-11T23:31:17.278+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:750d67aa:::obj572:head)]) -- 0x4d96d80 con 0x3752000 2026-02-11T23:31:17.278+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.279+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 579 ==== MOSDPGPush(1.0 111/109 [PushOp(1:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x389a4e0 con 0x3752000 2026-02-11T23:31:17.279+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.279+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.279+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 580 ==== pg_backfill(progress 1.0 e 111/111 lb 1:750d67aa:::obj572:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d94680 con 0x3752000 2026-02-11T23:31:17.279+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:750d67aa:::obj572:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 111 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.279+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:750d67aa:::obj572:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 111/109 [PushOp(1:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:750d67aa:::obj572:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:76c71d4e:::obj92:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:76c71d4e:::obj92:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:76c71d4e:::obj92:head 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:76c71d4e:::obj92:head v 107'92 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.279+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:750d67aa:::obj572:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.279+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:750d67aa:::obj572:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 111/111 lb 1:750d67aa:::obj572:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.279+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.279+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:750d67aa:::obj572:head) v3 2026-02-11T23:31:17.279+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7492d19a:::obj168:head local-lis/les=109/110 n=270 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:750d67aa:::obj572:head) 2026-02-11T23:31:17.279+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.279+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:750d67aa:::obj572:head) v3 finish latency 0.000015 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.279+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.279+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4a80 con 0x3752000 2026-02-11T23:31:17.279+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:76c71d4e:::obj92:head)]) -- 0x4d97080 con 0x3752000 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.279+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 581 ==== MOSDPGPush(1.0 111/109 [PushOp(1:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389a9c0 con 0x3752000 2026-02-11T23:31:17.279+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.279+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.279+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 582 ==== pg_backfill(progress 1.0 e 111/111 lb 1:76c71d4e:::obj92:head) ==== 1061+0+0 (unknown 0 0 0) 0x4d94c00 con 0x3752000 2026-02-11T23:31:17.279+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:76c71d4e:::obj92:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 111 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.279+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:76c71d4e:::obj92:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 111/109 [PushOp(1:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:76c71d4e:::obj92:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:76f9746b:::obj110:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:76f9746b:::obj110:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:76f9746b:::obj110:head 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:76f9746b:::obj110:head v 107'110 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.279+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.279+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.279+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:76c71d4e:::obj92:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.279+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:76c71d4e:::obj92:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 111/111 lb 1:76c71d4e:::obj92:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.279+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.279+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:76c71d4e:::obj92:head) v3 2026-02-11T23:31:17.279+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:750d67aa:::obj572:head local-lis/les=109/110 n=271 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:76c71d4e:::obj92:head) 2026-02-11T23:31:17.279+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.279+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:76c71d4e:::obj92:head) v3 finish latency 0.000015 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.279+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.280+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.280+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fea900 con 0x3752000 2026-02-11T23:31:17.280+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:76f9746b:::obj110:head)]) -- 0x4d97200 con 0x3752000 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.280+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 583 ==== MOSDPGPush(1.0 111/109 [PushOp(1:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4db6340 con 0x3752000 2026-02-11T23:31:17.280+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.280+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.280+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 584 ==== pg_backfill(progress 1.0 e 111/111 lb 1:76f9746b:::obj110:head) ==== 1062+0+0 (unknown 0 0 0) 0x4d95180 con 0x3752000 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.280+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:76f9746b:::obj110:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.280+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:76f9746b:::obj110:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.280+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 111/109 [PushOp(1:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.280+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.280+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.280+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.280+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:775ec905:::obj98:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:775ec905:::obj98:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.280+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:775ec905:::obj98:head 2026-02-11T23:31:17.280+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:775ec905:::obj98:head v 107'98 2026-02-11T23:31:17.280+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.280+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:76f9746b:::obj110:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.280+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:76f9746b:::obj110:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:76f9746b:::obj110:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 111/111 lb 1:76f9746b:::obj110:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:76f9746b:::obj110:head) v3 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76c71d4e:::obj92:head local-lis/les=109/110 n=272 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:76f9746b:::obj110:head) 2026-02-11T23:31:17.280+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:76f9746b:::obj110:head) v3 finish latency 0.000019 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.280+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.280+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fea600 con 0x3752000 2026-02-11T23:31:17.280+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:775ec905:::obj98:head)]) -- 0x4fe4900 con 0x3752000 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.280+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 585 ==== MOSDPGPush(1.0 111/109 [PushOp(1:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4db6820 con 0x3752000 2026-02-11T23:31:17.280+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 111 2026-02-11T23:31:17.280+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.280+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 586 ==== pg_backfill(progress 1.0 e 111/111 lb 1:775ec905:::obj98:head) ==== 1061+0+0 (unknown 0 0 0) 0x4d95700 con 0x3752000 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.280+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:775ec905:::obj98:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 111 2026-02-11T23:31:17.280+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:775ec905:::obj98:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 111/109 [PushOp(1:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.280+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:775ec905:::obj98:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:776b5b83:::obj306:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.280+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:776b5b83:::obj306:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:776b5b83:::obj306:head 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:776b5b83:::obj306:head v 107'306 2026-02-11T23:31:17.280+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.280+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.281+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.281+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:775ec905:::obj98:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:775ec905:::obj98:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 111/111 lb 1:775ec905:::obj98:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:775ec905:::obj98:head) v3 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:76f9746b:::obj110:head local-lis/les=109/110 n=273 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:775ec905:::obj98:head) 2026-02-11T23:31:17.281+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:775ec905:::obj98:head) v3 finish latency 0.000017 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.281+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.281+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4780 con 0x3752000 2026-02-11T23:31:17.281+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:776b5b83:::obj306:head)]) -- 0x4d97500 con 0x3752000 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.281+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 587 ==== MOSDPGPush(1.0 111/109 [PushOp(1:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4db6d00 con 0x3752000 2026-02-11T23:31:17.281+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.281+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.281+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.281+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 588 ==== pg_backfill(progress 1.0 e 111/111 lb 1:776b5b83:::obj306:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3752000 2026-02-11T23:31:17.281+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.281+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:776b5b83:::obj306:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.281+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:776b5b83:::obj306:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.281+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.281+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.281+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 111/109 [PushOp(1:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.281+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:776b5b83:::obj306:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.281+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.281+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.281+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:779a6cde:::obj377:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.281+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:779a6cde:::obj377:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.281+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:779a6cde:::obj377:head 2026-02-11T23:31:17.281+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:779a6cde:::obj377:head v 107'377 2026-02-11T23:31:17.281+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.281+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.281+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.281+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:776b5b83:::obj306:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:776b5b83:::obj306:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 111/111 lb 1:776b5b83:::obj306:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:776b5b83:::obj306:head) v3 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:775ec905:::obj98:head local-lis/les=109/110 n=274 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:776b5b83:::obj306:head) 2026-02-11T23:31:17.281+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:776b5b83:::obj306:head) v3 finish latency 0.000015 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.281+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.281+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.281+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fea480 con 0x3752000 2026-02-11T23:31:17.281+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:779a6cde:::obj377:head)]) -- 0x4d97800 con 0x3752000 2026-02-11T23:31:17.281+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.282+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 589 ==== MOSDPGPush(1.0 111/109 [PushOp(1:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4db71e0 con 0x3752000 2026-02-11T23:31:17.282+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.282+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.282+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 590 ==== pg_backfill(progress 1.0 e 111/111 lb 1:779a6cde:::obj377:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3752000 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.282+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:779a6cde:::obj377:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 111 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.282+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:779a6cde:::obj377:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 111/109 [PushOp(1:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:779a6cde:::obj377:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:786d07d6:::obj325:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:786d07d6:::obj325:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:786d07d6:::obj325:head 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:786d07d6:::obj325:head v 107'325 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.282+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:779a6cde:::obj377:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:779a6cde:::obj377:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 111/111 lb 1:779a6cde:::obj377:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:779a6cde:::obj377:head) v3 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:776b5b83:::obj306:head local-lis/les=109/110 n=275 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:779a6cde:::obj377:head) 2026-02-11T23:31:17.282+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:779a6cde:::obj377:head) v3 finish latency 0.000015 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.282+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4600 con 0x3752000 2026-02-11T23:31:17.282+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:786d07d6:::obj325:head)]) -- 0x4d97b00 con 0x3752000 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.282+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 591 ==== MOSDPGPush(1.0 111/109 [PushOp(1:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4db76c0 con 0x3752000 2026-02-11T23:31:17.282+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.282+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.282+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 592 ==== pg_backfill(progress 1.0 e 111/111 lb 1:786d07d6:::obj325:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3752000 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.282+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:786d07d6:::obj325:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 111 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.282+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:786d07d6:::obj325:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 111/109 [PushOp(1:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:786d07d6:::obj325:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:78a5b78c:::obj388:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:78a5b78c:::obj388:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:78a5b78c:::obj388:head 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:78a5b78c:::obj388:head v 107'388 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.282+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.282+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.282+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:786d07d6:::obj325:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:786d07d6:::obj325:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 111/111 lb 1:786d07d6:::obj325:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:786d07d6:::obj325:head) v3 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:779a6cde:::obj377:head local-lis/les=109/110 n=276 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:786d07d6:::obj325:head) 2026-02-11T23:31:17.282+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.282+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:786d07d6:::obj325:head) v3 finish latency 0.000012 2026-02-11T23:31:17.282+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.283+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.283+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.283+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fb9e00 con 0x3752000 2026-02-11T23:31:17.283+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:78a5b78c:::obj388:head)]) -- 0x4fea180 con 0x3752000 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.283+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 593 ==== MOSDPGPush(1.0 111/109 [PushOp(1:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4db7ba0 con 0x3752000 2026-02-11T23:31:17.283+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 111 2026-02-11T23:31:17.283+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.283+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.283+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 594 ==== pg_backfill(progress 1.0 e 111/111 lb 1:78a5b78c:::obj388:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3752000 2026-02-11T23:31:17.283+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:78a5b78c:::obj388:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 111 2026-02-11T23:31:17.283+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.283+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:78a5b78c:::obj388:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:78a5b78c:::obj388:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.283+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.283+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.283+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 111/109 [PushOp(1:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.283+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.283+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.283+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.283+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7903d8f8:::obj457:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.283+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7903d8f8:::obj457:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.283+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7903d8f8:::obj457:head 2026-02-11T23:31:17.283+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7903d8f8:::obj457:head v 107'457 2026-02-11T23:31:17.283+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.283+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.283+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.283+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:78a5b78c:::obj388:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.283+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:78a5b78c:::obj388:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 111/111 lb 1:78a5b78c:::obj388:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.283+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.283+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:78a5b78c:::obj388:head) v3 2026-02-11T23:31:17.283+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:786d07d6:::obj325:head local-lis/les=109/110 n=277 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:78a5b78c:::obj388:head) 2026-02-11T23:31:17.283+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.283+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:78a5b78c:::obj388:head) v3 finish latency 0.000014 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.283+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.283+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.283+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4300 con 0x3752000 2026-02-11T23:31:17.283+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:7903d8f8:::obj457:head)]) -- 0x4d97e00 con 0x3752000 2026-02-11T23:31:17.283+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.284+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 595 ==== MOSDPGPush(1.0 111/109 [PushOp(1:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dca1a0 con 0x3752000 2026-02-11T23:31:17.284+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.284+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.284+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 596 ==== pg_backfill(progress 1.0 e 111/111 lb 1:7903d8f8:::obj457:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3752000 2026-02-11T23:31:17.284+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:7903d8f8:::obj457:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.284+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7903d8f8:::obj457:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 111/109 [PushOp(1:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7903d8f8:::obj457:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:79ec8b1f:::obj508:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:79ec8b1f:::obj508:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:79ec8b1f:::obj508:head 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:79ec8b1f:::obj508:head v 107'508 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.284+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7903d8f8:::obj457:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7903d8f8:::obj457:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 111/111 lb 1:7903d8f8:::obj457:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:7903d8f8:::obj457:head) v3 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:78a5b78c:::obj388:head local-lis/les=109/110 n=278 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:7903d8f8:::obj457:head) 2026-02-11T23:31:17.284+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7903d8f8:::obj457:head) v3 finish latency 0.000013 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.284+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4000 con 0x3752000 2026-02-11T23:31:17.284+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:79ec8b1f:::obj508:head)]) -- 0x4dd0180 con 0x3752000 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.284+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 597 ==== MOSDPGPush(1.0 111/109 [PushOp(1:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dca680 con 0x3752000 2026-02-11T23:31:17.284+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.284+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.284+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 598 ==== pg_backfill(progress 1.0 e 111/111 lb 1:79ec8b1f:::obj508:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3752000 2026-02-11T23:31:17.284+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:79ec8b1f:::obj508:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 111 2026-02-11T23:31:17.284+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:79ec8b1f:::obj508:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 111/109 [PushOp(1:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:79ec8b1f:::obj508:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7a057f92:::obj247:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7a057f92:::obj247:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7a057f92:::obj247:head 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7a057f92:::obj247:head v 107'247 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.284+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.284+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:79ec8b1f:::obj508:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:79ec8b1f:::obj508:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 111/111 lb 1:79ec8b1f:::obj508:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:79ec8b1f:::obj508:head) v3 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7903d8f8:::obj457:head local-lis/les=109/110 n=279 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:79ec8b1f:::obj508:head) 2026-02-11T23:31:17.284+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:79ec8b1f:::obj508:head) v3 finish latency 0.000011 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.284+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.284+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.284+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fe4180 con 0x3752000 2026-02-11T23:31:17.284+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:7a057f92:::obj247:head)]) -- 0x4dd0480 con 0x3752000 2026-02-11T23:31:17.284+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.285+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 599 ==== MOSDPGPush(1.0 111/109 [PushOp(1:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcab60 con 0x3752000 2026-02-11T23:31:17.285+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 111 2026-02-11T23:31:17.285+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.285+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 600 ==== pg_backfill(progress 1.0 e 111/111 lb 1:7a057f92:::obj247:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3752000 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.285+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a057f92:::obj247:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 111 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.285+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a057f92:::obj247:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 111/109 [PushOp(1:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a057f92:::obj247:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7a539578:::obj175:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7a539578:::obj175:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7a539578:::obj175:head 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7a539578:::obj175:head v 107'175 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.285+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a057f92:::obj247:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a057f92:::obj247:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 111/111 lb 1:7a057f92:::obj247:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:7a057f92:::obj247:head) v3 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:79ec8b1f:::obj508:head local-lis/les=109/110 n=280 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:7a057f92:::obj247:head) 2026-02-11T23:31:17.285+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a057f92:::obj247:head) v3 finish latency 0.000015 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.285+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fb9b00 con 0x3752000 2026-02-11T23:31:17.285+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:7a539578:::obj175:head)]) -- 0x4dd0780 con 0x3752000 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.285+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 601 ==== MOSDPGPush(1.0 111/109 [PushOp(1:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcb040 con 0x3752000 2026-02-11T23:31:17.285+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 111 2026-02-11T23:31:17.285+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.285+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 602 ==== pg_backfill(progress 1.0 e 111/111 lb 1:7a539578:::obj175:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3752000 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.285+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a539578:::obj175:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 111 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.285+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a539578:::obj175:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000048 MOSDPGPush(1.0 111/109 [PushOp(1:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a539578:::obj175:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7a71687f:::obj592:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7a71687f:::obj592:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7a71687f:::obj592:head 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7a71687f:::obj592:head v 107'592 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.285+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.285+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.285+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a539578:::obj175:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a539578:::obj175:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 1.0 e 111/111 lb 1:7a539578:::obj175:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:7a539578:::obj175:head) v3 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a057f92:::obj247:head local-lis/les=109/110 n=281 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:7a539578:::obj175:head) 2026-02-11T23:31:17.285+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.285+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a539578:::obj175:head) v3 finish latency 0.000011 2026-02-11T23:31:17.285+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.286+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.286+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.286+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa1c80 con 0x3752000 2026-02-11T23:31:17.286+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:7a71687f:::obj592:head)]) -- 0x4fa1e00 con 0x3752000 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.286+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 603 ==== MOSDPGPush(1.0 111/109 [PushOp(1:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcb520 con 0x3752000 2026-02-11T23:31:17.286+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 111 2026-02-11T23:31:17.286+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.286+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.286+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.286+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.286+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 604 ==== pg_backfill(progress 1.0 e 111/111 lb 1:7a71687f:::obj592:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbec00 con 0x3752000 2026-02-11T23:31:17.286+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.286+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a71687f:::obj592:head) v3 prio 15 type 95 cost 0 latency 0.000015 epoch 111 2026-02-11T23:31:17.286+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a71687f:::obj592:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.286+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 111/109 [PushOp(1:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.286+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.286+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.286+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a71687f:::obj592:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.286+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7a94cb37:::obj345:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.286+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7a94cb37:::obj345:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.286+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7a94cb37:::obj345:head 2026-02-11T23:31:17.286+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7a94cb37:::obj345:head v 107'345 2026-02-11T23:31:17.286+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.286+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.286+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.286+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a71687f:::obj592:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.286+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a71687f:::obj592:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 111/111 lb 1:7a71687f:::obj592:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.286+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.286+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:7a71687f:::obj592:head) v3 2026-02-11T23:31:17.286+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a539578:::obj175:head local-lis/les=109/110 n=282 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:7a71687f:::obj592:head) 2026-02-11T23:31:17.286+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.286+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a71687f:::obj592:head) v3 finish latency 0.000014 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.286+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.286+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.286+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa1800 con 0x3752000 2026-02-11T23:31:17.286+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:7a94cb37:::obj345:head)]) -- 0x4dd0a80 con 0x3752000 2026-02-11T23:31:17.286+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.287+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 605 ==== MOSDPGPush(1.0 111/109 [PushOp(1:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3752000 2026-02-11T23:31:17.287+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 111 2026-02-11T23:31:17.287+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.287+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 606 ==== pg_backfill(progress 1.0 e 111/111 lb 1:7a94cb37:::obj345:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf180 con 0x3752000 2026-02-11T23:31:17.287+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a94cb37:::obj345:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 111 2026-02-11T23:31:17.287+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.287+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a94cb37:::obj345:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.287+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.287+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.287+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.287+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(1.0 111/109 [PushOp(1:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.287+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a94cb37:::obj345:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.287+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.287+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.287+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7a95c0e7:::obj245:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.287+0000 7fac51168640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7a95c0e7:::obj245:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.287+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7a95c0e7:::obj245:head 2026-02-11T23:31:17.287+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7a95c0e7:::obj245:head v 107'245 2026-02-11T23:31:17.287+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.287+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.287+0000 7fac51168640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000082 2026-02-11T23:31:17.287+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a94cb37:::obj345:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a94cb37:::obj345:head) v3 prio 15 cost 0 latency 0.000119 pg_backfill(progress 1.0 e 111/111 lb 1:7a94cb37:::obj345:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:7a94cb37:::obj345:head) v3 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a71687f:::obj592:head local-lis/les=109/110 n=283 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:7a94cb37:::obj345:head) 2026-02-11T23:31:17.287+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a94cb37:::obj345:head) v3 finish latency 0.000044 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.287+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.287+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fa1b00 con 0x3752000 2026-02-11T23:31:17.287+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:7a95c0e7:::obj245:head)]) -- 0x4dd0d80 con 0x3752000 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.287+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.287+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 607 ==== MOSDPGPush(1.0 111/109 [PushOp(1:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4000 con 0x3752000 2026-02-11T23:31:17.287+0000 7fac703fa640 15 osd.0 111 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000017 epoch 111 2026-02-11T23:31:17.287+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.287+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 608 ==== pg_backfill(progress 1.0 e 111/111 lb 1:7a95c0e7:::obj245:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3752000 2026-02-11T23:31:17.287+0000 7fac703fa640 15 osd.0 111 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a95c0e7:::obj245:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 111 2026-02-11T23:31:17.287+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a95c0e7:::obj245:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 111/109 [PushOp(1:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.287+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.287+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a95c0e7:::obj245:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7ac8ba2d:::obj506:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7ac8ba2d:::obj506:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7ac8ba2d:::obj506:head 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7ac8ba2d:::obj506:head v 107'506 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.287+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.287+0000 7fac55170640 10 osd.0 111 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.287+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7a95c0e7:::obj245:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a95c0e7:::obj245:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 111/111 lb 1:7a95c0e7:::obj245:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 111 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:7a95c0e7:::obj245:head) v3 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a94cb37:::obj345:head local-lis/les=109/110 n=284 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:7a95c0e7:::obj245:head) 2026-02-11T23:31:17.288+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 111 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7a95c0e7:::obj245:head) v3 finish latency 0.000032 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 111 get_con_osd_cluster to osd.1 from_epoch 111 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 111 get_nextmap_reserved map_reservations: {111=1} 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 111 release_map epoch: 111 2026-02-11T23:31:17.288+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e111/111) -- 0x4fb9200 con 0x3752000 2026-02-11T23:31:17.288+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:7ac8ba2d:::obj506:head)]) -- 0x4fb9500 con 0x3752000 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.288+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 40 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 112) e112/112) ==== 158+0+0 (unknown 0 0 0) 0x3f3f0e0 con 0x3e4a000 2026-02-11T23:31:17.288+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 39 ==== osd_map(112..112 src has 1..112) ==== 651+0+0 (unknown 0 0 0) 0x39416c0 con 0x3e4a000 2026-02-11T23:31:17.288+0000 7fac703fa640 15 osd.0 111 enqueue_peering_evt 4.0 epoch_sent: 112 epoch_requested: 112 MQuery 4.0 from 4 query_epoch 112 query: query(info 0'0 epoch_sent 112) 2026-02-11T23:31:17.288+0000 7fac606e4640 20 osd.0 111 OSD::ms_dispatch: osd_map(112..112 src has 1..112) 2026-02-11T23:31:17.288+0000 7fac606e4640 20 osd.0 111 _dispatch 0x39416c0 osd_map(112..112 src has 1..112) 2026-02-11T23:31:17.288+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 MQuery 4.0 from 4 query_epoch 112 query: query(info 0'0 epoch_sent 112)) class_id 2 prio 255 cost 10 e112) 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.288+0000 7fac606e4640 3 osd.0 111 handle_osd_map epochs [112,112], i have 111, src has [1,112] 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 MQuery 4.0 from 4 query_epoch 112 query: query(info 0'0 epoch_sent 112)) class_id 2immediate prio 255 cost 10 e112) queued 2026-02-11T23:31:17.288+0000 7fac606e4640 10 osd.0 111 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 MQuery 4.0 from 4 query_epoch 112 query: query(info 0'0 epoch_sent 112)) class_id 2immediate prio 255 cost 10 e112) pg 0 2026-02-11T23:31:17.288+0000 7fac606e4640 10 osd.0 111 handle_osd_map got inc map for epoch 112 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 map 112 > 111 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 112, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 MQuery 4.0 from 4 query_epoch 112 query: query(info 0'0 epoch_sent 112)) class_id 2immediate prio 255 cost 10 e112) 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.288+0000 7fac606e4640 20 osd.0 111 got_full_map 112, nothing requested 2026-02-11T23:31:17.288+0000 7fac606e4640 20 osd.0 111 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:17.288+0000 7fac606e4640 20 osd.0 111 handle_osd_map pg_num_history pg_num_history(e112 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:31:17.288+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {112={}} 2026-02-11T23:31:17.288+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:17.288+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e112 maps [1~112] lci=[5,112] tlb=1) 2026-02-11T23:31:17.288+0000 7fac5e983640 10 osd.0 111 _committed_osd_maps 112..112 2026-02-11T23:31:17.288+0000 7fac5e983640 10 osd.0 111 advance to epoch 112 (<= last 112 <= newest_map 112) 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0 112 consume_map version 112 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0 111 await_reserved_maps epoch:112 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0 111 await_reserved_maps done 0 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0 112 consume_map 112 -- publish done 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x49cc160 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 4.0 has no pg and waiting_for_split 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x25809a0 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0 112 identify_splits_and_merges 1.0 e111 to e112 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0 112 identify_splits_and_merges 1.0 e111 to e112 pg_nums {24=1} 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0 112 prune_pg_created 2026-02-11T23:31:17.288+0000 7fac5e983640 10 osd.0:0.consume_map 112 (was 111) 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 112 <= 112, requeueing 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {112=} 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:17.288+0000 7fac5e983640 10 osd.0:1.consume_map 112 (was 111) 2026-02-11T23:31:17.288+0000 7fac5e983640 10 osd.0:2.consume_map 112 (was 111) 2026-02-11T23:31:17.288+0000 7fac5e983640 10 osd.0:3.consume_map 112 (was 111) 2026-02-11T23:31:17.288+0000 7fac5e983640 10 osd.0:4.consume_map 112 (was 111) 2026-02-11T23:31:17.288+0000 7fac5e983640 10 osd.0:5.consume_map 112 (was 111) 2026-02-11T23:31:17.288+0000 7fac5e983640 10 osd.0:6.consume_map 112 (was 111) 2026-02-11T23:31:17.288+0000 7fac5e983640 10 osd.0:7.consume_map 112 (was 111) 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 MQuery 4.0 from 4 query_epoch 112 query: query(info 0'0 epoch_sent 112)) class_id 2immediate prio 255 cost 10 e112) queued 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 MQuery 4.0 from 4 query_epoch 112 query: query(info 0'0 epoch_sent 112)) class_id 2immediate prio 255 cost 10 e112) pg 0 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 112 handle_pg_query_nopg 4.0 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 112 pg 4.0 dne 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 112 get_con_osd_cluster to osd.4 from_epoch 112 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 111 -> 112 (shared) 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 112 send_incremental_map epoch range: (112, 112] to v2:127.0.0.1:6836/864727417 2026-02-11T23:31:17.288+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 609 ==== MOSDPGPush(1.0 111/109 [PushOp(1:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de44e0 con 0x3752000 2026-02-11T23:31:17.288+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 111/109 [PushOp(1:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 111 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 112 add_map_inc_bl 112 607 bytes 2026-02-11T23:31:17.288+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e111) 2026-02-11T23:31:17.288+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(112..112 src has 1..112) -- 0x3941dc0 con 0x3e4a000 2026-02-11T23:31:17.288+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:112 sent:112 4.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) ([0,0] all_participants={} intervals=[])) e112/112) -- 0x4dcea00 con 0x3e4a000 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.288+0000 7fac5e983640 10 osd.0 112 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.288+0000 7fac5e983640 15 osd.0 112 enqueue_peering_evt 1.0 epoch_sent: 112 epoch_requested: 112 NullEvt 2026-02-11T23:31:17.288+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 610 ==== pg_backfill(progress 1.0 e 111/111 lb 1:7ac8ba2d:::obj506:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec000 con 0x3752000 2026-02-11T23:31:17.288+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 NullEvt) class_id 2 prio 255 cost 10 e112) 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) queued 2026-02-11T23:31:17.288+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 111/111 lb 1:7ac8ba2d:::obj506:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 111 2026-02-11T23:31:17.288+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7ac8ba2d:::obj506:head) v3) class_id 0 prio 15 cost 0 e111) 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 111/109 [PushOp(1:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e111) pg 0x4a2a000 2026-02-11T23:31:17.288+0000 7fac5e983640 7 osd.0 112 activate_map version 112 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000064 MOSDPGPush(1.0 111/109 [PushOp(1:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.288+0000 7fac55971640 20 osd.0 112 agent_entry empty queue 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 111 is not newer than session's projected_epoch of 111 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 111/109 [PushOp(1:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 NullEvt) class_id 2immediate prio 255 cost 10 e112) queued 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7af66aa8:::obj163:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7af66aa8:::obj163:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7af66aa8:::obj163:head 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7af66aa8:::obj163:head v 107'163 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.288+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 111/109 [PushOp(1:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000082 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7ac8ba2d:::obj506:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) queued 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 112 epoch_requested: 112 NullEvt) class_id 2immediate prio 255 cost 10 e112) pg 0x4a2a000 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_advance_map: 112 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 111 -> 112 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 pg_epoch: 111 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1] -- 1/1 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] state: Started advmap 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 112 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_activate_map: epoch range: 112..112 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 111 while current is 112 2026-02-11T23:31:17.288+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.288+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 112 epoch_requested: 112 NullEvt 2026-02-11T23:31:17.288+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.288+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 111/111 lb 1:7ac8ba2d:::obj506:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e111) pg 0x4a2a000 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7ac8ba2d:::obj506:head) v3 prio 15 cost 0 latency 0.000148 pg_backfill(progress 1.0 e 111/111 lb 1:7ac8ba2d:::obj506:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 111 -> 112 (shared) 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 112 send_incremental_map epoch range: (112, 112] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:31:17.288+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(112..112 src has 1..112) -- 0x46eb180 con 0x3752000 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 111/111 lb 1:7ac8ba2d:::obj506:head) v3 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7a95c0e7:::obj245:head local-lis/les=109/110 n=285 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 111/111 lb 1:7ac8ba2d:::obj506:head) 2026-02-11T23:31:17.288+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.288+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 111/111 lb 1:7ac8ba2d:::obj506:head) v3 finish latency 0.000027 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.288+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.289+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.289+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.289+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa1380 con 0x3752000 2026-02-11T23:31:17.289+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 111/109 [PushReplyOp(1:7af66aa8:::obj163:head)]) -- 0x4fa1500 con 0x3752000 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.289+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.289+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 611 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de49c0 con 0x3752000 2026-02-11T23:31:17.289+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.289+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.289+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.289+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.289+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.289+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.289+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.289+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7b326036:::obj411:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7b326036:::obj411:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.289+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7b326036:::obj411:head 2026-02-11T23:31:17.289+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7b326036:::obj411:head v 107'411 2026-02-11T23:31:17.289+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.289+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.289+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 613 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7af66aa8:::obj163:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3752000 2026-02-11T23:31:17.289+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7af66aa8:::obj163:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.289+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 612 ==== osd_map(112..112 src has 1..112) ==== 651+0+0 (unknown 0 0 0) 0x46eb180 con 0x3752000 2026-02-11T23:31:17.289+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7af66aa8:::obj163:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.289+0000 7fac606e4640 20 osd.0 112 OSD::ms_dispatch: osd_map(112..112 src has 1..112) 2026-02-11T23:31:17.289+0000 7fac606e4640 20 osd.0 112 _dispatch 0x46eb180 osd_map(112..112 src has 1..112) 2026-02-11T23:31:17.289+0000 7fac606e4640 3 osd.0 112 handle_osd_map epochs [112,112], i have 112, src has [1,112] 2026-02-11T23:31:17.289+0000 7fac606e4640 10 osd.0 112 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.289+0000 7fac606e4640 10 osd.0 112 no new maps here, dropping 2026-02-11T23:31:17.289+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.289+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7af66aa8:::obj163:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.289+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.289+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.289+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7af66aa8:::obj163:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.289+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7af66aa8:::obj163:head) v3 prio 15 cost 0 latency 0.000035 pg_backfill(progress 1.0 e 112/112 lb 1:7af66aa8:::obj163:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.289+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.289+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7af66aa8:::obj163:head) v3 2026-02-11T23:31:17.289+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ac8ba2d:::obj506:head local-lis/les=109/110 n=286 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7af66aa8:::obj163:head) 2026-02-11T23:31:17.289+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.289+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7af66aa8:::obj163:head) v3 finish latency 0.000021 2026-02-11T23:31:17.289+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.289+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.289+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.289+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fb8900 con 0x3752000 2026-02-11T23:31:17.289+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7b326036:::obj411:head)]) -- 0x4fb8c00 con 0x3752000 2026-02-11T23:31:17.289+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.290+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 614 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4ea0 con 0x3752000 2026-02-11T23:31:17.290+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 112 2026-02-11T23:31:17.290+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.290+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 615 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7b326036:::obj411:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3752000 2026-02-11T23:31:17.290+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7b326036:::obj411:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.290+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7b326036:::obj411:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 112/109 [PushOp(1:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7b326036:::obj411:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7c121b42:::obj116:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7c121b42:::obj116:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7c121b42:::obj116:head 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7c121b42:::obj116:head v 107'116 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.290+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7b326036:::obj411:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7b326036:::obj411:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 112/112 lb 1:7b326036:::obj411:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7b326036:::obj411:head) v3 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7af66aa8:::obj163:head local-lis/les=109/110 n=287 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7b326036:::obj411:head) 2026-02-11T23:31:17.290+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7b326036:::obj411:head) v3 finish latency 0.000032 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.290+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4780 con 0x3752000 2026-02-11T23:31:17.290+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7c121b42:::obj116:head)]) -- 0x4dd1080 con 0x3752000 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.290+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 616 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de5380 con 0x3752000 2026-02-11T23:31:17.290+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.290+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.290+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 617 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7c121b42:::obj116:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded080 con 0x3752000 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.290+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c121b42:::obj116:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.290+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c121b42:::obj116:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 112/109 [PushOp(1:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c121b42:::obj116:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7c3a2962:::obj277:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7c3a2962:::obj277:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7c3a2962:::obj277:head 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7c3a2962:::obj277:head v 107'277 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.290+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.290+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.290+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c121b42:::obj116:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c121b42:::obj116:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 112/112 lb 1:7c121b42:::obj116:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7c121b42:::obj116:head) v3 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7b326036:::obj411:head local-lis/les=109/110 n=288 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7c121b42:::obj116:head) 2026-02-11T23:31:17.290+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.290+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c121b42:::obj116:head) v3 finish latency 0.000025 2026-02-11T23:31:17.290+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.291+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.291+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d97200 con 0x3752000 2026-02-11T23:31:17.291+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7c3a2962:::obj277:head)]) -- 0x4fe4900 con 0x3752000 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.291+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.291+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 618 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de5860 con 0x3752000 2026-02-11T23:31:17.291+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.291+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.291+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 619 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7c3a2962:::obj277:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3752000 2026-02-11T23:31:17.291+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c3a2962:::obj277:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.291+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c3a2962:::obj277:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.291+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.291+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c3a2962:::obj277:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7c406887:::obj566:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7c406887:::obj566:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7c406887:::obj566:head 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7c406887:::obj566:head v 107'566 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.291+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.291+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.291+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c3a2962:::obj277:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.291+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c3a2962:::obj277:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 112/112 lb 1:7c3a2962:::obj277:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.291+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.291+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7c3a2962:::obj277:head) v3 2026-02-11T23:31:17.291+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c121b42:::obj116:head local-lis/les=109/110 n=289 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7c3a2962:::obj277:head) 2026-02-11T23:31:17.291+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.291+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c3a2962:::obj277:head) v3 finish latency 0.000027 2026-02-11T23:31:17.291+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.291+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.291+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4a80 con 0x3752000 2026-02-11T23:31:17.291+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7c406887:::obj566:head)]) -- 0x4fea900 con 0x3752000 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.291+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.291+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 620 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de5d40 con 0x3752000 2026-02-11T23:31:17.291+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.291+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.291+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 621 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7c406887:::obj566:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3752000 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.291+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c406887:::obj566:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.291+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c406887:::obj566:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.291+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.291+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c406887:::obj566:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.291+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7c955dfc:::obj322:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.291+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7c955dfc:::obj322:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7c955dfc:::obj322:head 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7c955dfc:::obj322:head v 107'322 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.292+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.292+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.292+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c406887:::obj566:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.292+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c406887:::obj566:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 112/112 lb 1:7c406887:::obj566:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.292+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.292+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7c406887:::obj566:head) v3 2026-02-11T23:31:17.292+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c3a2962:::obj277:head local-lis/les=109/110 n=290 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7c406887:::obj566:head) 2026-02-11T23:31:17.292+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.292+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c406887:::obj566:head) v3 finish latency 0.000031 2026-02-11T23:31:17.292+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.292+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.292+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96a80 con 0x3752000 2026-02-11T23:31:17.292+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7c955dfc:::obj322:head)]) -- 0x4d96d80 con 0x3752000 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.292+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.292+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 622 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dfe340 con 0x3752000 2026-02-11T23:31:17.292+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.292+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.292+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 623 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7c955dfc:::obj322:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3752000 2026-02-11T23:31:17.292+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c955dfc:::obj322:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.292+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c955dfc:::obj322:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.292+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.292+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c955dfc:::obj322:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7ccaea2b:::obj486:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7ccaea2b:::obj486:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7ccaea2b:::obj486:head 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7ccaea2b:::obj486:head v 107'486 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.292+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.292+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.292+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7c955dfc:::obj322:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.292+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c955dfc:::obj322:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 112/112 lb 1:7c955dfc:::obj322:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.292+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.292+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7c955dfc:::obj322:head) v3 2026-02-11T23:31:17.292+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c406887:::obj566:head local-lis/les=109/110 n=291 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7c955dfc:::obj322:head) 2026-02-11T23:31:17.292+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.292+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7c955dfc:::obj322:head) v3 finish latency 0.000024 2026-02-11T23:31:17.292+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.292+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.292+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.292+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4feaf00 con 0x3752000 2026-02-11T23:31:17.292+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7ccaea2b:::obj486:head)]) -- 0x4fe4c00 con 0x3752000 2026-02-11T23:31:17.292+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.293+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 624 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dfe820 con 0x3752000 2026-02-11T23:31:17.293+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.293+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.293+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 625 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7ccaea2b:::obj486:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3752000 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.293+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ccaea2b:::obj486:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 112 2026-02-11T23:31:17.293+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ccaea2b:::obj486:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ccaea2b:::obj486:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7e5f2af9:::obj239:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7e5f2af9:::obj239:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7e5f2af9:::obj239:head 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7e5f2af9:::obj239:head v 107'239 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.293+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ccaea2b:::obj486:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.293+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ccaea2b:::obj486:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 112/112 lb 1:7ccaea2b:::obj486:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.293+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.293+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7ccaea2b:::obj486:head) v3 2026-02-11T23:31:17.293+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7c955dfc:::obj322:head local-lis/les=109/110 n=292 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7ccaea2b:::obj486:head) 2026-02-11T23:31:17.293+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.293+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ccaea2b:::obj486:head) v3 finish latency 0.000024 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.293+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96180 con 0x3752000 2026-02-11T23:31:17.293+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7e5f2af9:::obj239:head)]) -- 0x4d96480 con 0x3752000 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.293+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 626 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4dfed00 con 0x3752000 2026-02-11T23:31:17.293+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.293+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.293+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 627 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7e5f2af9:::obj239:head) ==== 1062+0+0 (unknown 0 0 0) 0x4deec00 con 0x3752000 2026-02-11T23:31:17.293+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7e5f2af9:::obj239:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.293+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7e5f2af9:::obj239:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 112/109 [PushOp(1:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7e5f2af9:::obj239:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7eacede5:::obj24:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7eacede5:::obj24:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7eacede5:::obj24:head 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7eacede5:::obj24:head v 107'24 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.293+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.293+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.293+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7e5f2af9:::obj239:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.293+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7e5f2af9:::obj239:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 112/112 lb 1:7e5f2af9:::obj239:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.293+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.293+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7e5f2af9:::obj239:head) v3 2026-02-11T23:31:17.293+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ccaea2b:::obj486:head local-lis/les=109/110 n=293 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7e5f2af9:::obj239:head) 2026-02-11T23:31:17.293+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.293+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7e5f2af9:::obj239:head) v3 finish latency 0.000024 2026-02-11T23:31:17.293+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.294+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.294+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.294+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96000 con 0x3752000 2026-02-11T23:31:17.294+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7eacede5:::obj24:head)]) -- 0x4fe5e00 con 0x3752000 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.294+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 628 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dff1e0 con 0x3752000 2026-02-11T23:31:17.294+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.294+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.294+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 629 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7eacede5:::obj24:head) ==== 1061+0+0 (unknown 0 0 0) 0x4def180 con 0x3752000 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.294+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7eacede5:::obj24:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.294+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7eacede5:::obj24:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.294+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.294+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.294+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.294+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.294+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7eacede5:::obj24:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.294+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.294+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7ec5cfba:::obj597:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7ec5cfba:::obj597:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.294+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7ec5cfba:::obj597:head 2026-02-11T23:31:17.294+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7ec5cfba:::obj597:head v 107'597 2026-02-11T23:31:17.294+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.294+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.294+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.294+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.294+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7eacede5:::obj24:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.294+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7eacede5:::obj24:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 112/112 lb 1:7eacede5:::obj24:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.294+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.294+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7eacede5:::obj24:head) v3 2026-02-11T23:31:17.294+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7e5f2af9:::obj239:head local-lis/les=109/110 n=294 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7eacede5:::obj24:head) 2026-02-11T23:31:17.294+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.294+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7eacede5:::obj24:head) v3 finish latency 0.000026 2026-02-11T23:31:17.294+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.294+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.294+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.294+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5b00 con 0x3752000 2026-02-11T23:31:17.294+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7ec5cfba:::obj597:head)]) -- 0x4fe5c80 con 0x3752000 2026-02-11T23:31:17.294+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.295+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 630 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dff6c0 con 0x3752000 2026-02-11T23:31:17.295+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.295+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.295+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 631 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7ec5cfba:::obj597:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def700 con 0x3752000 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.295+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ec5cfba:::obj597:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.295+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ec5cfba:::obj597:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 112/109 [PushOp(1:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ec5cfba:::obj597:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7ee98ed8:::obj216:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7ee98ed8:::obj216:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7ee98ed8:::obj216:head 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7ee98ed8:::obj216:head v 107'216 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.295+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ec5cfba:::obj597:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.295+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ec5cfba:::obj597:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 112/112 lb 1:7ec5cfba:::obj597:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.295+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.295+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7ec5cfba:::obj597:head) v3 2026-02-11T23:31:17.295+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7eacede5:::obj24:head local-lis/les=109/110 n=295 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7ec5cfba:::obj597:head) 2026-02-11T23:31:17.295+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.295+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ec5cfba:::obj597:head) v3 finish latency 0.000025 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.295+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4febb00 con 0x3752000 2026-02-11T23:31:17.295+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7ee98ed8:::obj216:head)]) -- 0x4fe5800 con 0x3752000 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.295+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 632 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dffba0 con 0x3752000 2026-02-11T23:31:17.295+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.295+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.295+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 633 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7ee98ed8:::obj216:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e16000 con 0x3752000 2026-02-11T23:31:17.295+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ee98ed8:::obj216:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.295+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ee98ed8:::obj216:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ee98ed8:::obj216:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7ee9cafa:::obj105:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7ee9cafa:::obj105:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7ee9cafa:::obj105:head 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7ee9cafa:::obj105:head v 107'105 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.295+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ee98ed8:::obj216:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.295+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ee98ed8:::obj216:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 112/112 lb 1:7ee98ed8:::obj216:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.295+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.295+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7ee98ed8:::obj216:head) v3 2026-02-11T23:31:17.295+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ec5cfba:::obj597:head local-lis/les=109/110 n=296 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7ee98ed8:::obj216:head) 2026-02-11T23:31:17.295+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.295+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ee98ed8:::obj216:head) v3 finish latency 0.000027 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.295+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.295+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5500 con 0x3752000 2026-02-11T23:31:17.295+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7ee9cafa:::obj105:head)]) -- 0x4fe5680 con 0x3752000 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.295+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.295+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.296+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 634 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e1a1a0 con 0x3752000 2026-02-11T23:31:17.296+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.296+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.296+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 635 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7ee9cafa:::obj105:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e16580 con 0x3752000 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.296+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ee9cafa:::obj105:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.296+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ee9cafa:::obj105:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 112/109 [PushOp(1:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ee9cafa:::obj105:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7f12fc77:::obj195:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7f12fc77:::obj195:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7f12fc77:::obj195:head 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7f12fc77:::obj195:head v 107'195 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.296+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7ee9cafa:::obj105:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.296+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ee9cafa:::obj105:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 112/112 lb 1:7ee9cafa:::obj105:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.296+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.296+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7ee9cafa:::obj105:head) v3 2026-02-11T23:31:17.296+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee98ed8:::obj216:head local-lis/les=109/110 n=297 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7ee9cafa:::obj105:head) 2026-02-11T23:31:17.296+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.296+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7ee9cafa:::obj105:head) v3 finish latency 0.000023 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.296+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4f00 con 0x3752000 2026-02-11T23:31:17.296+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7f12fc77:::obj195:head)]) -- 0x4feb500 con 0x3752000 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.296+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 636 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e1a680 con 0x3752000 2026-02-11T23:31:17.296+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.296+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.296+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 637 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7f12fc77:::obj195:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e16b00 con 0x3752000 2026-02-11T23:31:17.296+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7f12fc77:::obj195:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.296+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7f12fc77:::obj195:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 112/109 [PushOp(1:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7f12fc77:::obj195:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7f1dde7e:::obj491:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7f1dde7e:::obj491:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7f1dde7e:::obj491:head 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7f1dde7e:::obj491:head v 107'491 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.296+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.296+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.296+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7f12fc77:::obj195:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.296+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7f12fc77:::obj195:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 112/112 lb 1:7f12fc77:::obj195:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.296+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.296+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7f12fc77:::obj195:head) v3 2026-02-11T23:31:17.296+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7ee9cafa:::obj105:head local-lis/les=109/110 n=298 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7f12fc77:::obj195:head) 2026-02-11T23:31:17.296+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.296+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7f12fc77:::obj195:head) v3 finish latency 0.000022 2026-02-11T23:31:17.296+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.297+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.297+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.297+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4a80 con 0x3752000 2026-02-11T23:31:17.297+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7f1dde7e:::obj491:head)]) -- 0x4fe5200 con 0x3752000 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.297+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 638 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e1ab60 con 0x3752000 2026-02-11T23:31:17.297+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.297+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.297+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 639 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7f1dde7e:::obj491:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e17080 con 0x3752000 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.297+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7f1dde7e:::obj491:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.297+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7f1dde7e:::obj491:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.297+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 112/109 [PushOp(1:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.297+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.297+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.297+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7f1dde7e:::obj491:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.297+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.297+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.297+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7f2a7765:::obj252:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7f2a7765:::obj252:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.297+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7f2a7765:::obj252:head 2026-02-11T23:31:17.297+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7f2a7765:::obj252:head v 107'252 2026-02-11T23:31:17.297+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.297+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.297+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.297+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.297+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7f1dde7e:::obj491:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.297+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7f1dde7e:::obj491:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 112/112 lb 1:7f1dde7e:::obj491:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.297+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.297+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7f1dde7e:::obj491:head) v3 2026-02-11T23:31:17.297+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f12fc77:::obj195:head local-lis/les=109/110 n=299 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7f1dde7e:::obj491:head) 2026-02-11T23:31:17.297+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.297+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7f1dde7e:::obj491:head) v3 finish latency 0.000030 2026-02-11T23:31:17.297+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.297+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.297+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.297+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd1080 con 0x3752000 2026-02-11T23:31:17.297+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7f2a7765:::obj252:head)]) -- 0x4fe4900 con 0x3752000 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.297+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.297+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.297+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 640 ==== MOSDPGPush(1.0 112/109 [PushOp(1:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e1b040 con 0x3752000 2026-02-11T23:31:17.297+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.298+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.298+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 641 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7f2a7765:::obj252:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e17600 con 0x3752000 2026-02-11T23:31:17.298+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7f2a7765:::obj252:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.298+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7f2a7765:::obj252:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7f2a7765:::obj252:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:7fffd9c5:::obj553:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:7fffd9c5:::obj553:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:7fffd9c5:::obj553:head 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:7fffd9c5:::obj553:head v 107'553 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.298+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7f2a7765:::obj252:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.298+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7f2a7765:::obj252:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 112/112 lb 1:7f2a7765:::obj252:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.298+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.298+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7f2a7765:::obj252:head) v3 2026-02-11T23:31:17.298+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f1dde7e:::obj491:head local-lis/les=109/110 n=300 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7f2a7765:::obj252:head) 2026-02-11T23:31:17.298+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.298+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7f2a7765:::obj252:head) v3 finish latency 0.000013 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.298+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fb8900 con 0x3752000 2026-02-11T23:31:17.298+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:7fffd9c5:::obj553:head)]) -- 0x4fe4780 con 0x3752000 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.298+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 642 ==== MOSDPGPush(1.0 112/109 [PushOp(1:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4e1b520 con 0x3752000 2026-02-11T23:31:17.298+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.298+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.298+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 643 ==== pg_backfill(progress 1.0 e 112/112 lb 1:7fffd9c5:::obj553:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e17b80 con 0x3752000 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.298+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:7fffd9c5:::obj553:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.298+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7fffd9c5:::obj553:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7fffd9c5:::obj553:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:81617860:::obj28:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:81617860:::obj28:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:81617860:::obj28:head 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:81617860:::obj28:head v 107'28 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.298+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:7fffd9c5:::obj553:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.298+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7fffd9c5:::obj553:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 112/112 lb 1:7fffd9c5:::obj553:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.298+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.298+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:7fffd9c5:::obj553:head) v3 2026-02-11T23:31:17.298+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7f2a7765:::obj252:head local-lis/les=109/110 n=301 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:7fffd9c5:::obj553:head) 2026-02-11T23:31:17.298+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.298+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:7fffd9c5:::obj553:head) v3 finish latency 0.000012 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.298+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.298+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.298+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa1380 con 0x3752000 2026-02-11T23:31:17.298+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:81617860:::obj28:head)]) -- 0x4fa1500 con 0x3752000 2026-02-11T23:31:17.298+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.299+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 644 ==== MOSDPGPush(1.0 112/109 [PushOp(1:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e1ba00 con 0x3752000 2026-02-11T23:31:17.299+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.299+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.299+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 645 ==== pg_backfill(progress 1.0 e 112/112 lb 1:81617860:::obj28:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e18100 con 0x3752000 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.299+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:81617860:::obj28:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.299+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:81617860:::obj28:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.299+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.299+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:81617860:::obj28:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:81fa42e1:::obj447:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:81fa42e1:::obj447:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:81fa42e1:::obj447:head 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:81fa42e1:::obj447:head v 107'447 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.299+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.299+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.299+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:81617860:::obj28:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.299+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:81617860:::obj28:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 1.0 e 112/112 lb 1:81617860:::obj28:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.299+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.299+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:81617860:::obj28:head) v3 2026-02-11T23:31:17.299+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:7fffd9c5:::obj553:head local-lis/les=109/110 n=302 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:81617860:::obj28:head) 2026-02-11T23:31:17.299+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.299+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:81617860:::obj28:head) v3 finish latency 0.000012 2026-02-11T23:31:17.299+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.299+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.299+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd0d80 con 0x3752000 2026-02-11T23:31:17.299+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:81fa42e1:::obj447:head)]) -- 0x4fb9500 con 0x3752000 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.299+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 646 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e2e000 con 0x3752000 2026-02-11T23:31:17.299+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.299+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.299+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 647 ==== pg_backfill(progress 1.0 e 112/112 lb 1:81fa42e1:::obj447:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e18680 con 0x3752000 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.299+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:81fa42e1:::obj447:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.299+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:81fa42e1:::obj447:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.299+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.299+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.299+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:81fa42e1:::obj447:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8205b5d8:::obj400:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8205b5d8:::obj400:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8205b5d8:::obj400:head 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8205b5d8:::obj400:head v 107'400 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.299+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.299+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.299+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.299+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.299+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:81fa42e1:::obj447:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.299+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:81fa42e1:::obj447:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 112/112 lb 1:81fa42e1:::obj447:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.299+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.299+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:81fa42e1:::obj447:head) v3 2026-02-11T23:31:17.299+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81617860:::obj28:head local-lis/les=109/110 n=303 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:81fa42e1:::obj447:head) 2026-02-11T23:31:17.299+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.300+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:81fa42e1:::obj447:head) v3 finish latency 0.000014 2026-02-11T23:31:17.300+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.300+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.300+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.300+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa1800 con 0x3752000 2026-02-11T23:31:17.300+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8205b5d8:::obj400:head)]) -- 0x4fa1b00 con 0x3752000 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.300+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 648 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e2e4e0 con 0x3752000 2026-02-11T23:31:17.300+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.300+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.300+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 649 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8205b5d8:::obj400:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e18c00 con 0x3752000 2026-02-11T23:31:17.300+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8205b5d8:::obj400:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.300+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8205b5d8:::obj400:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.300+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 112/109 [PushOp(1:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.300+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.300+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.300+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8205b5d8:::obj400:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.300+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.300+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.300+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8250b831:::obj124:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8250b831:::obj124:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.300+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8250b831:::obj124:head 2026-02-11T23:31:17.300+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8250b831:::obj124:head v 107'124 2026-02-11T23:31:17.300+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.300+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.300+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.300+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.300+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8205b5d8:::obj400:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.300+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8205b5d8:::obj400:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 112/112 lb 1:8205b5d8:::obj400:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.300+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.300+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8205b5d8:::obj400:head) v3 2026-02-11T23:31:17.300+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:81fa42e1:::obj447:head local-lis/les=109/110 n=304 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8205b5d8:::obj400:head) 2026-02-11T23:31:17.300+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.300+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8205b5d8:::obj400:head) v3 finish latency 0.000015 2026-02-11T23:31:17.300+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.300+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.300+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.300+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd0780 con 0x3752000 2026-02-11T23:31:17.300+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8250b831:::obj124:head)]) -- 0x4fa1e00 con 0x3752000 2026-02-11T23:31:17.300+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.301+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 650 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e2e9c0 con 0x3752000 2026-02-11T23:31:17.301+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.301+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.301+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 651 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8250b831:::obj124:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e19180 con 0x3752000 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.301+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8250b831:::obj124:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.301+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8250b831:::obj124:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 112/109 [PushOp(1:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8250b831:::obj124:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8260b93f:::obj338:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8260b93f:::obj338:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8260b93f:::obj338:head 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8260b93f:::obj338:head v 107'338 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.301+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8250b831:::obj124:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.301+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8250b831:::obj124:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 112/112 lb 1:8250b831:::obj124:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.301+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.301+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8250b831:::obj124:head) v3 2026-02-11T23:31:17.301+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8205b5d8:::obj400:head local-lis/les=109/110 n=305 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8250b831:::obj124:head) 2026-02-11T23:31:17.301+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.301+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8250b831:::obj124:head) v3 finish latency 0.000012 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.301+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4180 con 0x3752000 2026-02-11T23:31:17.301+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8260b93f:::obj338:head)]) -- 0x4fb9b00 con 0x3752000 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.301+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 652 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e2eea0 con 0x3752000 2026-02-11T23:31:17.301+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.301+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.301+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 653 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8260b93f:::obj338:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e19700 con 0x3752000 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.301+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8260b93f:::obj338:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.301+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8260b93f:::obj338:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 112/109 [PushOp(1:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8260b93f:::obj338:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8280535c:::obj236:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8280535c:::obj236:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8280535c:::obj236:head 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8280535c:::obj236:head v 107'236 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.301+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8260b93f:::obj338:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.301+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8260b93f:::obj338:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 112/112 lb 1:8260b93f:::obj338:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.301+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.301+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8260b93f:::obj338:head) v3 2026-02-11T23:31:17.301+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8250b831:::obj124:head local-lis/les=109/110 n=306 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8260b93f:::obj338:head) 2026-02-11T23:31:17.301+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.301+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8260b93f:::obj338:head) v3 finish latency 0.000015 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.301+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.301+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.301+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d97e00 con 0x3752000 2026-02-11T23:31:17.301+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8280535c:::obj236:head)]) -- 0x4dd0180 con 0x3752000 2026-02-11T23:31:17.301+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.302+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 654 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e2f380 con 0x3752000 2026-02-11T23:31:17.302+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.302+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.302+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.302+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 655 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8280535c:::obj236:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e42000 con 0x3752000 2026-02-11T23:31:17.302+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8280535c:::obj236:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.302+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8280535c:::obj236:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.302+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.302+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.302+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.302+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.302+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8280535c:::obj236:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.302+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.302+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.302+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8421ae49:::obj196:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.302+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8421ae49:::obj196:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.302+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8421ae49:::obj196:head 2026-02-11T23:31:17.302+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8421ae49:::obj196:head v 107'196 2026-02-11T23:31:17.302+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.302+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.302+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:17.302+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8280535c:::obj236:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8280535c:::obj236:head) v3 prio 15 cost 0 latency 0.000102 pg_backfill(progress 1.0 e 112/112 lb 1:8280535c:::obj236:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8280535c:::obj236:head) v3 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8260b93f:::obj338:head local-lis/les=109/110 n=307 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8280535c:::obj236:head) 2026-02-11T23:31:17.302+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8280535c:::obj236:head) v3 finish latency 0.000018 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.302+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.302+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4300 con 0x3752000 2026-02-11T23:31:17.302+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8421ae49:::obj196:head)]) -- 0x4dd1380 con 0x3752000 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.302+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 656 ==== MOSDPGPush(1.0 112/109 [PushOp(1:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e2f860 con 0x3752000 2026-02-11T23:31:17.302+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.302+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.302+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 657 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8421ae49:::obj196:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e42580 con 0x3752000 2026-02-11T23:31:17.302+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8421ae49:::obj196:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.302+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8421ae49:::obj196:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.302+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.302+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8421ae49:::obj196:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:84855744:::obj451:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.302+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:84855744:::obj451:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:84855744:::obj451:head 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:84855744:::obj451:head v 107'451 2026-02-11T23:31:17.302+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.302+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.303+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.303+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8421ae49:::obj196:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.303+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8421ae49:::obj196:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 1.0 e 112/112 lb 1:8421ae49:::obj196:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.303+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.303+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8421ae49:::obj196:head) v3 2026-02-11T23:31:17.303+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8280535c:::obj236:head local-lis/les=109/110 n=308 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8421ae49:::obj196:head) 2026-02-11T23:31:17.303+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.303+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8421ae49:::obj196:head) v3 finish latency 0.000019 2026-02-11T23:31:17.303+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.303+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.303+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d97b00 con 0x3752000 2026-02-11T23:31:17.303+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:84855744:::obj451:head)]) -- 0x4fea180 con 0x3752000 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.303+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 658 ==== MOSDPGPush(1.0 112/109 [PushOp(1:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4e2fd40 con 0x3752000 2026-02-11T23:31:17.303+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.303+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.303+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 659 ==== pg_backfill(progress 1.0 e 112/112 lb 1:84855744:::obj451:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e42b00 con 0x3752000 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.303+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:84855744:::obj451:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.303+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:84855744:::obj451:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 112/109 [PushOp(1:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.303+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.303+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:84855744:::obj451:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:84c1344d:::obj97:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:84c1344d:::obj97:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:84c1344d:::obj97:head 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:84c1344d:::obj97:head v 107'97 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.303+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.303+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.303+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:84855744:::obj451:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.303+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:84855744:::obj451:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 1.0 e 112/112 lb 1:84855744:::obj451:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.303+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.303+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:84855744:::obj451:head) v3 2026-02-11T23:31:17.303+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8421ae49:::obj196:head local-lis/les=109/110 n=309 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:84855744:::obj451:head) 2026-02-11T23:31:17.303+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.303+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:84855744:::obj451:head) v3 finish latency 0.000013 2026-02-11T23:31:17.303+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.303+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.303+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.303+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d97800 con 0x3752000 2026-02-11T23:31:17.303+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:84c1344d:::obj97:head)]) -- 0x4fe4600 con 0x3752000 2026-02-11T23:31:17.303+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.304+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 660 ==== MOSDPGPush(1.0 112/109 [PushOp(1:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e46340 con 0x3752000 2026-02-11T23:31:17.304+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.304+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.304+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 661 ==== pg_backfill(progress 1.0 e 112/112 lb 1:84c1344d:::obj97:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e43080 con 0x3752000 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.304+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:84c1344d:::obj97:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.304+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:84c1344d:::obj97:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:84c1344d:::obj97:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:86bb8a56:::obj416:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:86bb8a56:::obj416:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:86bb8a56:::obj416:head 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:86bb8a56:::obj416:head v 107'416 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.304+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:84c1344d:::obj97:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:84c1344d:::obj97:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 112/112 lb 1:84c1344d:::obj97:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:84c1344d:::obj97:head) v3 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84855744:::obj451:head local-lis/les=109/110 n=310 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:84c1344d:::obj97:head) 2026-02-11T23:31:17.304+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:84c1344d:::obj97:head) v3 finish latency 0.000017 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.304+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fea480 con 0x3752000 2026-02-11T23:31:17.304+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:86bb8a56:::obj416:head)]) -- 0x4dd1680 con 0x3752000 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.304+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 662 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e46820 con 0x3752000 2026-02-11T23:31:17.304+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.304+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.304+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 663 ==== pg_backfill(progress 1.0 e 112/112 lb 1:86bb8a56:::obj416:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e43600 con 0x3752000 2026-02-11T23:31:17.304+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:86bb8a56:::obj416:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.304+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:86bb8a56:::obj416:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 112/109 [PushOp(1:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:86bb8a56:::obj416:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8745f65b:::obj476:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8745f65b:::obj476:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8745f65b:::obj476:head 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8745f65b:::obj476:head v 107'476 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.304+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:86bb8a56:::obj416:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:86bb8a56:::obj416:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 112/112 lb 1:86bb8a56:::obj416:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:86bb8a56:::obj416:head) v3 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:84c1344d:::obj97:head local-lis/les=109/110 n=311 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:86bb8a56:::obj416:head) 2026-02-11T23:31:17.304+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.304+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:86bb8a56:::obj416:head) v3 finish latency 0.000018 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.304+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.304+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.304+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa1e00 con 0x3752000 2026-02-11T23:31:17.304+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8745f65b:::obj476:head)]) -- 0x4d97500 con 0x3752000 2026-02-11T23:31:17.304+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.305+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 664 ==== MOSDPGPush(1.0 112/109 [PushOp(1:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4e46d00 con 0x3752000 2026-02-11T23:31:17.305+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.305+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.305+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 665 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8745f65b:::obj476:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e43b80 con 0x3752000 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.305+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8745f65b:::obj476:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.305+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8745f65b:::obj476:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.305+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.305+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8745f65b:::obj476:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:87eb7818:::obj91:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:87eb7818:::obj91:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:87eb7818:::obj91:head 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:87eb7818:::obj91:head v 107'91 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.305+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.305+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.305+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8745f65b:::obj476:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.305+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8745f65b:::obj476:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 112/112 lb 1:8745f65b:::obj476:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.305+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.305+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8745f65b:::obj476:head) v3 2026-02-11T23:31:17.305+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:86bb8a56:::obj416:head local-lis/les=109/110 n=312 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8745f65b:::obj476:head) 2026-02-11T23:31:17.305+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.305+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8745f65b:::obj476:head) v3 finish latency 0.000015 2026-02-11T23:31:17.305+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.305+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.305+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa1b00 con 0x3752000 2026-02-11T23:31:17.305+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:87eb7818:::obj91:head)]) -- 0x4dd0780 con 0x3752000 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.305+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 666 ==== MOSDPGPush(1.0 112/109 [PushOp(1:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e471e0 con 0x3752000 2026-02-11T23:31:17.305+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.305+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.305+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 667 ==== pg_backfill(progress 1.0 e 112/112 lb 1:87eb7818:::obj91:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e44100 con 0x3752000 2026-02-11T23:31:17.305+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:87eb7818:::obj91:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.305+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:87eb7818:::obj91:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.305+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.305+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:87eb7818:::obj91:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 112/109 [PushOp(1:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:881499a5:::obj460:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.305+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:881499a5:::obj460:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:881499a5:::obj460:head 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:881499a5:::obj460:head v 107'460 2026-02-11T23:31:17.305+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.305+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:87eb7818:::obj91:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.306+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:87eb7818:::obj91:head) v3 prio 15 cost 0 latency 0.000103 pg_backfill(progress 1.0 e 112/112 lb 1:87eb7818:::obj91:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.306+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.306+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:87eb7818:::obj91:head) v3 2026-02-11T23:31:17.306+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8745f65b:::obj476:head local-lis/les=109/110 n=313 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:87eb7818:::obj91:head) 2026-02-11T23:31:17.306+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.306+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:87eb7818:::obj91:head) v3 finish latency 0.000021 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.306+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd0d80 con 0x3752000 2026-02-11T23:31:17.306+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:881499a5:::obj460:head)]) -- 0x4fa1800 con 0x3752000 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.306+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 668 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e476c0 con 0x3752000 2026-02-11T23:31:17.306+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.306+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.306+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 669 ==== pg_backfill(progress 1.0 e 112/112 lb 1:881499a5:::obj460:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e44680 con 0x3752000 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.306+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:881499a5:::obj460:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 112 2026-02-11T23:31:17.306+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:881499a5:::obj460:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:881499a5:::obj460:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8835249a:::obj346:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8835249a:::obj346:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8835249a:::obj346:head 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8835249a:::obj346:head v 107'346 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.306+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:881499a5:::obj460:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.306+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:881499a5:::obj460:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 1.0 e 112/112 lb 1:881499a5:::obj460:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.306+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.306+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:881499a5:::obj460:head) v3 2026-02-11T23:31:17.306+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:87eb7818:::obj91:head local-lis/les=109/110 n=314 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:881499a5:::obj460:head) 2026-02-11T23:31:17.306+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.306+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:881499a5:::obj460:head) v3 finish latency 0.000016 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.306+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.306+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.306+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4780 con 0x3752000 2026-02-11T23:31:17.306+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8835249a:::obj346:head)]) -- 0x4fa1500 con 0x3752000 2026-02-11T23:31:17.306+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.307+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 670 ==== MOSDPGPush(1.0 112/109 [PushOp(1:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e47ba0 con 0x3752000 2026-02-11T23:31:17.307+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.307+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.307+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 671 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8835249a:::obj346:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e44c00 con 0x3752000 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.307+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8835249a:::obj346:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.307+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8835249a:::obj346:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8835249a:::obj346:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:88c90a82:::obj574:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:88c90a82:::obj574:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:88c90a82:::obj574:head 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:88c90a82:::obj574:head v 107'574 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.307+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8835249a:::obj346:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8835249a:::obj346:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 112/112 lb 1:8835249a:::obj346:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8835249a:::obj346:head) v3 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:881499a5:::obj460:head local-lis/les=109/110 n=315 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8835249a:::obj346:head) 2026-02-11T23:31:17.307+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8835249a:::obj346:head) v3 finish latency 0.000029 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.307+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fb8900 con 0x3752000 2026-02-11T23:31:17.307+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:88c90a82:::obj574:head)]) -- 0x4dd1980 con 0x3752000 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.307+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 672 ==== MOSDPGPush(1.0 112/109 [PushOp(1:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4e5e1a0 con 0x3752000 2026-02-11T23:31:17.307+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.307+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.307+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 673 ==== pg_backfill(progress 1.0 e 112/112 lb 1:88c90a82:::obj574:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e45180 con 0x3752000 2026-02-11T23:31:17.307+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:88c90a82:::obj574:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.307+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:88c90a82:::obj574:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:88c90a82:::obj574:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:88e6b0fd:::obj99:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:88e6b0fd:::obj99:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:88e6b0fd:::obj99:head 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:88e6b0fd:::obj99:head v 107'99 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.307+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.307+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:88c90a82:::obj574:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:88c90a82:::obj574:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 112/112 lb 1:88c90a82:::obj574:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:88c90a82:::obj574:head) v3 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8835249a:::obj346:head local-lis/les=109/110 n=316 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:88c90a82:::obj574:head) 2026-02-11T23:31:17.307+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:88c90a82:::obj574:head) v3 finish latency 0.000016 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.307+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.307+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.307+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4900 con 0x3752000 2026-02-11T23:31:17.307+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:88e6b0fd:::obj99:head)]) -- 0x4dd1c80 con 0x3752000 2026-02-11T23:31:17.307+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.308+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 674 ==== MOSDPGPush(1.0 112/109 [PushOp(1:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e5e680 con 0x3752000 2026-02-11T23:31:17.308+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.308+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.308+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.308+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 675 ==== pg_backfill(progress 1.0 e 112/112 lb 1:88e6b0fd:::obj99:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e45700 con 0x3752000 2026-02-11T23:31:17.308+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:88e6b0fd:::obj99:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.308+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.308+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:88e6b0fd:::obj99:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.308+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.308+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.308+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 112/109 [PushOp(1:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.308+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.308+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.308+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.308+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.308+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:88e6b0fd:::obj99:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.308+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:89565a01:::obj330:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.308+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:89565a01:::obj330:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.308+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:89565a01:::obj330:head 2026-02-11T23:31:17.308+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:89565a01:::obj330:head v 107'330 2026-02-11T23:31:17.308+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.308+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.308+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.308+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.308+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.308+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.308+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:88e6b0fd:::obj99:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.308+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:88e6b0fd:::obj99:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 112/112 lb 1:88e6b0fd:::obj99:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.308+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.308+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:88e6b0fd:::obj99:head) v3 2026-02-11T23:31:17.308+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88c90a82:::obj574:head local-lis/les=109/110 n=317 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:88e6b0fd:::obj99:head) 2026-02-11T23:31:17.308+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.308+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:88e6b0fd:::obj99:head) v3 finish latency 0.000015 2026-02-11T23:31:17.308+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.308+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.308+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.308+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.308+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.308+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.308+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4a80 con 0x3752000 2026-02-11T23:31:17.308+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:89565a01:::obj330:head)]) -- 0x4dd1080 con 0x3752000 2026-02-11T23:31:17.308+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.309+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 676 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e5eb60 con 0x3752000 2026-02-11T23:31:17.309+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.309+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.309+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 677 ==== pg_backfill(progress 1.0 e 112/112 lb 1:89565a01:::obj330:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3752000 2026-02-11T23:31:17.309+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:89565a01:::obj330:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.309+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:89565a01:::obj330:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:89565a01:::obj330:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(1.0 112/109 [PushOp(1:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8a5b4a06:::obj393:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8a5b4a06:::obj393:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8a5b4a06:::obj393:head 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8a5b4a06:::obj393:head v 107'393 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.309+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:89565a01:::obj330:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:89565a01:::obj330:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 112/112 lb 1:89565a01:::obj330:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:89565a01:::obj330:head) v3 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:88e6b0fd:::obj99:head local-lis/les=109/110 n=318 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:89565a01:::obj330:head) 2026-02-11T23:31:17.309+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:89565a01:::obj330:head) v3 finish latency 0.000020 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.309+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4feb500 con 0x3752000 2026-02-11T23:31:17.309+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8a5b4a06:::obj393:head)]) -- 0x4dd1e00 con 0x3752000 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.309+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 678 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e5f040 con 0x3752000 2026-02-11T23:31:17.309+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.309+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.309+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 679 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8a5b4a06:::obj393:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3752000 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.309+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8a5b4a06:::obj393:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.309+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8a5b4a06:::obj393:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8a5b4a06:::obj393:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8a7d5b8a:::obj217:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8a7d5b8a:::obj217:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8a7d5b8a:::obj217:head 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8a7d5b8a:::obj217:head v 107'217 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.309+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.309+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.309+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8a5b4a06:::obj393:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8a5b4a06:::obj393:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 112/112 lb 1:8a5b4a06:::obj393:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8a5b4a06:::obj393:head) v3 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:89565a01:::obj330:head local-lis/les=109/110 n=319 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8a5b4a06:::obj393:head) 2026-02-11T23:31:17.309+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.309+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8a5b4a06:::obj393:head) v3 finish latency 0.000015 2026-02-11T23:31:17.309+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.310+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.310+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.310+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5500 con 0x3752000 2026-02-11T23:31:17.310+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8a7d5b8a:::obj217:head)]) -- 0x4fe4f00 con 0x3752000 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.310+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 680 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e5f520 con 0x3752000 2026-02-11T23:31:17.310+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.310+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.310+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 681 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8a7d5b8a:::obj217:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3752000 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.310+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8a7d5b8a:::obj217:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.310+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8a7d5b8a:::obj217:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.310+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.310+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.310+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.310+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.310+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8a7d5b8a:::obj217:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.310+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.310+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8aa18807:::obj353:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8aa18807:::obj353:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.310+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8aa18807:::obj353:head 2026-02-11T23:31:17.310+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8aa18807:::obj353:head v 107'353 2026-02-11T23:31:17.310+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.310+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.310+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.310+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.310+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8a7d5b8a:::obj217:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.310+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8a7d5b8a:::obj217:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 112/112 lb 1:8a7d5b8a:::obj217:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.310+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.310+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8a7d5b8a:::obj217:head) v3 2026-02-11T23:31:17.310+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a5b4a06:::obj393:head local-lis/les=109/110 n=320 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8a7d5b8a:::obj217:head) 2026-02-11T23:31:17.310+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.310+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8a7d5b8a:::obj217:head) v3 finish latency 0.000015 2026-02-11T23:31:17.310+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.310+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.310+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.310+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5c80 con 0x3752000 2026-02-11T23:31:17.310+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8aa18807:::obj353:head)]) -- 0x4fe5800 con 0x3752000 2026-02-11T23:31:17.310+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.311+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 682 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e5fa00 con 0x3752000 2026-02-11T23:31:17.311+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.311+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.311+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 683 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8aa18807:::obj353:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d080 con 0x3752000 2026-02-11T23:31:17.311+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8aa18807:::obj353:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.311+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8aa18807:::obj353:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8aa18807:::obj353:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8b0915dc:::obj278:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8b0915dc:::obj278:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8b0915dc:::obj278:head 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8b0915dc:::obj278:head v 107'278 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.311+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8aa18807:::obj353:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8aa18807:::obj353:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 112/112 lb 1:8aa18807:::obj353:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8aa18807:::obj353:head) v3 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8a7d5b8a:::obj217:head local-lis/les=109/110 n=321 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8aa18807:::obj353:head) 2026-02-11T23:31:17.311+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8aa18807:::obj353:head) v3 finish latency 0.000018 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.311+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5b00 con 0x3752000 2026-02-11T23:31:17.311+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8b0915dc:::obj278:head)]) -- 0x4e70180 con 0x3752000 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.311+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 684 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7e000 con 0x3752000 2026-02-11T23:31:17.311+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.311+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.311+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 685 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8b0915dc:::obj278:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3752000 2026-02-11T23:31:17.311+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b0915dc:::obj278:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.311+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b0915dc:::obj278:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b0915dc:::obj278:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8b38e8a9:::obj385:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8b38e8a9:::obj385:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8b38e8a9:::obj385:head 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8b38e8a9:::obj385:head v 107'385 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.311+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b0915dc:::obj278:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b0915dc:::obj278:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 112/112 lb 1:8b0915dc:::obj278:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8b0915dc:::obj278:head) v3 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8aa18807:::obj353:head local-lis/les=109/110 n=322 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8b0915dc:::obj278:head) 2026-02-11T23:31:17.311+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.311+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b0915dc:::obj278:head) v3 finish latency 0.000021 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.311+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.311+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.311+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96480 con 0x3752000 2026-02-11T23:31:17.311+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8b38e8a9:::obj385:head)]) -- 0x4fe5e00 con 0x3752000 2026-02-11T23:31:17.311+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.312+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 686 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7e4e0 con 0x3752000 2026-02-11T23:31:17.312+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.312+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.312+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.312+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.312+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 687 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8b38e8a9:::obj385:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3752000 2026-02-11T23:31:17.312+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b38e8a9:::obj385:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.312+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.312+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b38e8a9:::obj385:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.312+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.312+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.312+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b38e8a9:::obj385:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.312+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.312+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.312+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8b716ab6:::obj324:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.312+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8b716ab6:::obj324:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.312+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8b716ab6:::obj324:head 2026-02-11T23:31:17.312+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8b716ab6:::obj324:head v 107'324 2026-02-11T23:31:17.312+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.312+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.312+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.312+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b38e8a9:::obj385:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b38e8a9:::obj385:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 112/112 lb 1:8b38e8a9:::obj385:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8b38e8a9:::obj385:head) v3 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b0915dc:::obj278:head local-lis/les=109/110 n=323 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8b38e8a9:::obj385:head) 2026-02-11T23:31:17.312+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b38e8a9:::obj385:head) v3 finish latency 0.000018 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.312+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.312+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96180 con 0x3752000 2026-02-11T23:31:17.312+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8b716ab6:::obj324:head)]) -- 0x4e70480 con 0x3752000 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.312+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 688 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7e9c0 con 0x3752000 2026-02-11T23:31:17.312+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.312+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.312+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 689 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8b716ab6:::obj324:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3752000 2026-02-11T23:31:17.312+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.312+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b716ab6:::obj324:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.312+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b716ab6:::obj324:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.312+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.312+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b716ab6:::obj324:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8b7a7bb6:::obj294:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.312+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8b7a7bb6:::obj294:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8b7a7bb6:::obj294:head 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8b7a7bb6:::obj294:head v 107'294 2026-02-11T23:31:17.312+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.312+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.313+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.313+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.313+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b716ab6:::obj324:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b716ab6:::obj324:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 112/112 lb 1:8b716ab6:::obj324:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8b716ab6:::obj324:head) v3 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b38e8a9:::obj385:head local-lis/les=109/110 n=324 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8b716ab6:::obj324:head) 2026-02-11T23:31:17.313+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b716ab6:::obj324:head) v3 finish latency 0.000018 2026-02-11T23:31:17.313+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.313+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.313+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.313+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96d80 con 0x3752000 2026-02-11T23:31:17.313+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8b7a7bb6:::obj294:head)]) -- 0x4fe4c00 con 0x3752000 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.313+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 690 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7eea0 con 0x3752000 2026-02-11T23:31:17.313+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000070 epoch 112 2026-02-11T23:31:17.313+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.313+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.313+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 691 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8b7a7bb6:::obj294:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3752000 2026-02-11T23:31:17.313+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.313+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b7a7bb6:::obj294:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.313+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b7a7bb6:::obj294:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.313+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.313+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000097 MOSDPGPush(1.0 112/109 [PushOp(1:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b7a7bb6:::obj294:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8c123081:::obj473:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.313+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8c123081:::obj473:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8c123081:::obj473:head 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8c123081:::obj473:head v 107'473 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.313+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.313+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.313+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8b7a7bb6:::obj294:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.313+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b7a7bb6:::obj294:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 112/112 lb 1:8b7a7bb6:::obj294:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.313+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.313+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8b7a7bb6:::obj294:head) v3 2026-02-11T23:31:17.313+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b716ab6:::obj324:head local-lis/les=109/110 n=325 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8b7a7bb6:::obj294:head) 2026-02-11T23:31:17.313+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.313+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8b7a7bb6:::obj294:head) v3 finish latency 0.000018 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.313+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.313+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.313+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96a80 con 0x3752000 2026-02-11T23:31:17.313+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8c123081:::obj473:head)]) -- 0x4e70780 con 0x3752000 2026-02-11T23:31:17.313+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.314+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 692 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f380 con 0x3752000 2026-02-11T23:31:17.314+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.314+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.314+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 693 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8c123081:::obj473:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3752000 2026-02-11T23:31:17.314+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8c123081:::obj473:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.314+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.314+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8c123081:::obj473:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.314+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.314+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.314+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.314+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.314+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.314+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8c123081:::obj473:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.314+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.314+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.314+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.314+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8c741e19:::obj599:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.314+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8c741e19:::obj599:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.314+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8c741e19:::obj599:head 2026-02-11T23:31:17.314+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8c741e19:::obj599:head v 107'599 2026-02-11T23:31:17.314+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.314+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.314+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.314+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.314+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.314+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8c123081:::obj473:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.314+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8c123081:::obj473:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 112/112 lb 1:8c123081:::obj473:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.314+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.314+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8c123081:::obj473:head) v3 2026-02-11T23:31:17.314+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8b7a7bb6:::obj294:head local-lis/les=109/110 n=326 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8c123081:::obj473:head) 2026-02-11T23:31:17.314+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.314+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8c123081:::obj473:head) v3 finish latency 0.000032 2026-02-11T23:31:17.314+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.314+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.314+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.314+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.314+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.314+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.314+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.314+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fea900 con 0x3752000 2026-02-11T23:31:17.314+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8c741e19:::obj599:head)]) -- 0x4e70a80 con 0x3752000 2026-02-11T23:31:17.314+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.314+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.314+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.315+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 694 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f860 con 0x3752000 2026-02-11T23:31:17.315+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 112 2026-02-11T23:31:17.315+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.315+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 695 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8c741e19:::obj599:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f180 con 0x3752000 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.315+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8c741e19:::obj599:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.315+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8c741e19:::obj599:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 112/109 [PushOp(1:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8c741e19:::obj599:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8ca4bb13:::obj342:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8ca4bb13:::obj342:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8ca4bb13:::obj342:head 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8ca4bb13:::obj342:head v 107'342 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.315+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8c741e19:::obj599:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8c741e19:::obj599:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 112/112 lb 1:8c741e19:::obj599:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8c741e19:::obj599:head) v3 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c123081:::obj473:head local-lis/les=109/110 n=327 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8c741e19:::obj599:head) 2026-02-11T23:31:17.315+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8c741e19:::obj599:head) v3 finish latency 0.000015 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.315+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5b00 con 0x3752000 2026-02-11T23:31:17.315+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8ca4bb13:::obj342:head)]) -- 0x4e70d80 con 0x3752000 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.315+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 696 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4e7fd40 con 0x3752000 2026-02-11T23:31:17.315+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000016 epoch 112 2026-02-11T23:31:17.315+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.315+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 697 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8ca4bb13:::obj342:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3752000 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.315+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8ca4bb13:::obj342:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.315+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8ca4bb13:::obj342:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 112/109 [PushOp(1:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8ca4bb13:::obj342:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8cdc434f:::obj77:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8cdc434f:::obj77:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8cdc434f:::obj77:head 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8cdc434f:::obj77:head v 107'77 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.315+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.315+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8ca4bb13:::obj342:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8ca4bb13:::obj342:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 112/112 lb 1:8ca4bb13:::obj342:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8ca4bb13:::obj342:head) v3 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8c741e19:::obj599:head local-lis/les=109/110 n=328 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8ca4bb13:::obj342:head) 2026-02-11T23:31:17.315+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8ca4bb13:::obj342:head) v3 finish latency 0.000014 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.315+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.315+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.315+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5800 con 0x3752000 2026-02-11T23:31:17.315+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8cdc434f:::obj77:head)]) -- 0x4e71080 con 0x3752000 2026-02-11T23:31:17.315+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.316+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 698 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5112340 con 0x3752000 2026-02-11T23:31:17.316+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 112 2026-02-11T23:31:17.316+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.316+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.316+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.316+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 699 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8cdc434f:::obj77:head) ==== 1061+0+0 (unknown 0 0 0) 0x511a000 con 0x3752000 2026-02-11T23:31:17.316+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.316+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8cdc434f:::obj77:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.316+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.316+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8cdc434f:::obj77:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.316+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.316+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 112/109 [PushOp(1:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.316+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8cdc434f:::obj77:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.316+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.316+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.316+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.316+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8d29890f:::obj305:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.316+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8d29890f:::obj305:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.316+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8d29890f:::obj305:head 2026-02-11T23:31:17.316+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8d29890f:::obj305:head v 107'305 2026-02-11T23:31:17.316+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.316+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.316+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.316+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.316+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.316+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8cdc434f:::obj77:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.316+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8cdc434f:::obj77:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 112/112 lb 1:8cdc434f:::obj77:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.316+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.316+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8cdc434f:::obj77:head) v3 2026-02-11T23:31:17.316+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8ca4bb13:::obj342:head local-lis/les=109/110 n=329 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8cdc434f:::obj77:head) 2026-02-11T23:31:17.316+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.316+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8cdc434f:::obj77:head) v3 finish latency 0.000014 2026-02-11T23:31:17.316+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.316+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.316+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.316+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.316+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.316+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.316+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.316+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5c80 con 0x3752000 2026-02-11T23:31:17.316+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8d29890f:::obj305:head)]) -- 0x4e71200 con 0x3752000 2026-02-11T23:31:17.316+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.316+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 700 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5112820 con 0x3752000 2026-02-11T23:31:17.316+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 112 2026-02-11T23:31:17.316+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.316+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.316+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 701 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8d29890f:::obj305:head) ==== 1062+0+0 (unknown 0 0 0) 0x511a580 con 0x3752000 2026-02-11T23:31:17.316+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8d29890f:::obj305:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.316+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.316+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8d29890f:::obj305:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.316+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.316+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8d29890f:::obj305:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 112/109 [PushOp(1:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8d384ce8:::obj297:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.317+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8d384ce8:::obj297:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8d384ce8:::obj297:head 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8d384ce8:::obj297:head v 107'297 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.317+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.317+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8d29890f:::obj305:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8d29890f:::obj305:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 112/112 lb 1:8d29890f:::obj305:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8d29890f:::obj305:head) v3 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8cdc434f:::obj77:head local-lis/les=109/110 n=330 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8d29890f:::obj305:head) 2026-02-11T23:31:17.317+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8d29890f:::obj305:head) v3 finish latency 0.000013 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.317+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.317+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4f00 con 0x3752000 2026-02-11T23:31:17.317+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8d384ce8:::obj297:head)]) -- 0x4e71500 con 0x3752000 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.317+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 702 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5112d00 con 0x3752000 2026-02-11T23:31:17.317+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000026 epoch 112 2026-02-11T23:31:17.317+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.317+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.317+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.317+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 703 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8d384ce8:::obj297:head) ==== 1062+0+0 (unknown 0 0 0) 0x511ab00 con 0x3752000 2026-02-11T23:31:17.317+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8d384ce8:::obj297:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.317+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.317+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8d384ce8:::obj297:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.317+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000063 MOSDPGPush(1.0 112/109 [PushOp(1:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8d6bd12e:::obj368:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8d384ce8:::obj297:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.317+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8d6bd12e:::obj368:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8d6bd12e:::obj368:head 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8d6bd12e:::obj368:head v 107'368 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.317+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.317+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.317+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8d384ce8:::obj297:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8d384ce8:::obj297:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 112/112 lb 1:8d384ce8:::obj297:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8d384ce8:::obj297:head) v3 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d29890f:::obj305:head local-lis/les=109/110 n=331 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8d384ce8:::obj297:head) 2026-02-11T23:31:17.317+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8d384ce8:::obj297:head) v3 finish latency 0.000013 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.317+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.317+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.317+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5500 con 0x3752000 2026-02-11T23:31:17.317+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8d6bd12e:::obj368:head)]) -- 0x4e71800 con 0x3752000 2026-02-11T23:31:17.317+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.318+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 704 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x51131e0 con 0x3752000 2026-02-11T23:31:17.318+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.318+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.318+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 705 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8d6bd12e:::obj368:head) ==== 1062+0+0 (unknown 0 0 0) 0x511b080 con 0x3752000 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.318+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8d6bd12e:::obj368:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 112 2026-02-11T23:31:17.318+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8d6bd12e:::obj368:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8d6bd12e:::obj368:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8dd16f86:::obj3:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8dd16f86:::obj3:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8dd16f86:::obj3:head 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8dd16f86:::obj3:head v 107'3 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.318+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8d6bd12e:::obj368:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8d6bd12e:::obj368:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 112/112 lb 1:8d6bd12e:::obj368:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8d6bd12e:::obj368:head) v3 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d384ce8:::obj297:head local-lis/les=109/110 n=332 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8d6bd12e:::obj368:head) 2026-02-11T23:31:17.318+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8d6bd12e:::obj368:head) v3 finish latency 0.000012 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.318+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd1e00 con 0x3752000 2026-02-11T23:31:17.318+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8dd16f86:::obj3:head)]) -- 0x4e71b00 con 0x3752000 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.318+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 706 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51136c0 con 0x3752000 2026-02-11T23:31:17.318+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.318+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.318+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 707 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8dd16f86:::obj3:head) ==== 1060+0+0 (unknown 0 0 0) 0x511b600 con 0x3752000 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.318+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8dd16f86:::obj3:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 112 2026-02-11T23:31:17.318+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8dd16f86:::obj3:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 112/109 [PushOp(1:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8dd16f86:::obj3:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8e988874:::obj200:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8e988874:::obj200:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8e988874:::obj200:head 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8e988874:::obj200:head v 107'200 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.318+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.318+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000075 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.318+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8dd16f86:::obj3:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8dd16f86:::obj3:head) v3 prio 15 cost 0 latency 0.000107 pg_backfill(progress 1.0 e 112/112 lb 1:8dd16f86:::obj3:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8dd16f86:::obj3:head) v3 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8d6bd12e:::obj368:head local-lis/les=109/110 n=333 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8dd16f86:::obj3:head) 2026-02-11T23:31:17.318+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.318+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8dd16f86:::obj3:head) v3 finish latency 0.000013 2026-02-11T23:31:17.318+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.319+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.319+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.319+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4a80 con 0x3752000 2026-02-11T23:31:17.319+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8e988874:::obj200:head)]) -- 0x4feb500 con 0x3752000 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.319+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 708 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5113ba0 con 0x3752000 2026-02-11T23:31:17.319+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 112 2026-02-11T23:31:17.319+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.319+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.319+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.319+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 709 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8e988874:::obj200:head) ==== 1062+0+0 (unknown 0 0 0) 0x511bb80 con 0x3752000 2026-02-11T23:31:17.319+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.319+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.319+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8e988874:::obj200:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 112 2026-02-11T23:31:17.319+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8e988874:::obj200:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.319+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8e988874:::obj200:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.319+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.319+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.319+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.319+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8eb14ea4:::obj384:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.319+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8eb14ea4:::obj384:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.319+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8eb14ea4:::obj384:head 2026-02-11T23:31:17.319+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8eb14ea4:::obj384:head v 107'384 2026-02-11T23:31:17.319+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.319+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.319+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.319+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8e988874:::obj200:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.319+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8e988874:::obj200:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 112/112 lb 1:8e988874:::obj200:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.319+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.319+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8e988874:::obj200:head) v3 2026-02-11T23:31:17.319+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8dd16f86:::obj3:head local-lis/les=109/110 n=334 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8e988874:::obj200:head) 2026-02-11T23:31:17.319+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.319+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8e988874:::obj200:head) v3 finish latency 0.000016 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.319+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.319+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.319+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd1c80 con 0x3752000 2026-02-11T23:31:17.319+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8eb14ea4:::obj384:head)]) -- 0x4e71c80 con 0x3752000 2026-02-11T23:31:17.319+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.320+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 710 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x512e1a0 con 0x3752000 2026-02-11T23:31:17.320+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 112 2026-02-11T23:31:17.320+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.320+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 711 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8eb14ea4:::obj384:head) ==== 1062+0+0 (unknown 0 0 0) 0x511c100 con 0x3752000 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.320+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8eb14ea4:::obj384:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 112 2026-02-11T23:31:17.320+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8eb14ea4:::obj384:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 112/109 [PushOp(1:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8eb14ea4:::obj384:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8f4c4fa9:::obj590:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8f4c4fa9:::obj590:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8f4c4fa9:::obj590:head 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8f4c4fa9:::obj590:head v 107'590 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.320+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8eb14ea4:::obj384:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8eb14ea4:::obj384:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 112/112 lb 1:8eb14ea4:::obj384:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8eb14ea4:::obj384:head) v3 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8e988874:::obj200:head local-lis/les=109/110 n=335 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8eb14ea4:::obj384:head) 2026-02-11T23:31:17.320+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8eb14ea4:::obj384:head) v3 finish latency 0.000012 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.320+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4900 con 0x3752000 2026-02-11T23:31:17.320+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8f4c4fa9:::obj590:head)]) -- 0x4fa1080 con 0x3752000 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.320+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 712 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x512e680 con 0x3752000 2026-02-11T23:31:17.320+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.320+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.320+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 713 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8f4c4fa9:::obj590:head) ==== 1062+0+0 (unknown 0 0 0) 0x511c680 con 0x3752000 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.320+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8f4c4fa9:::obj590:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.320+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8f4c4fa9:::obj590:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 112/109 [PushOp(1:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8f4c4fa9:::obj590:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8f7cdb2a:::obj538:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8f7cdb2a:::obj538:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8f7cdb2a:::obj538:head 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8f7cdb2a:::obj538:head v 107'538 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.320+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.320+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8f4c4fa9:::obj590:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8f4c4fa9:::obj590:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 112/112 lb 1:8f4c4fa9:::obj590:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8f4c4fa9:::obj590:head) v3 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8eb14ea4:::obj384:head local-lis/les=109/110 n=336 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8f4c4fa9:::obj590:head) 2026-02-11T23:31:17.320+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8f4c4fa9:::obj590:head) v3 finish latency 0.000013 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.320+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.320+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.320+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd1980 con 0x3752000 2026-02-11T23:31:17.320+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8f7cdb2a:::obj538:head)]) -- 0x4fa0f00 con 0x3752000 2026-02-11T23:31:17.320+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.321+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 714 ==== MOSDPGPush(1.0 112/109 [PushOp(1:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x512eb60 con 0x3752000 2026-02-11T23:31:17.321+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 112 2026-02-11T23:31:17.321+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.321+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 715 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8f7cdb2a:::obj538:head) ==== 1062+0+0 (unknown 0 0 0) 0x511cc00 con 0x3752000 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.321+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8f7cdb2a:::obj538:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.321+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8f7cdb2a:::obj538:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8f7cdb2a:::obj538:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:8fcfcc34:::obj221:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:8fcfcc34:::obj221:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:8fcfcc34:::obj221:head 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:8fcfcc34:::obj221:head v 107'221 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.321+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8f7cdb2a:::obj538:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.321+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8f7cdb2a:::obj538:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 112/112 lb 1:8f7cdb2a:::obj538:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.321+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.321+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8f7cdb2a:::obj538:head) v3 2026-02-11T23:31:17.321+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f4c4fa9:::obj590:head local-lis/les=109/110 n=337 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8f7cdb2a:::obj538:head) 2026-02-11T23:31:17.321+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.321+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8f7cdb2a:::obj538:head) v3 finish latency 0.000012 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.321+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.321+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fb8900 con 0x3752000 2026-02-11T23:31:17.321+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:8fcfcc34:::obj221:head)]) -- 0x4fa0d80 con 0x3752000 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.321+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 716 ==== MOSDPGPush(1.0 112/109 [PushOp(1:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x512f040 con 0x3752000 2026-02-11T23:31:17.321+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 112 2026-02-11T23:31:17.321+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.321+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 717 ==== pg_backfill(progress 1.0 e 112/112 lb 1:8fcfcc34:::obj221:head) ==== 1062+0+0 (unknown 0 0 0) 0x511d180 con 0x3752000 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.321+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:8fcfcc34:::obj221:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.321+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8fcfcc34:::obj221:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 112/109 [PushOp(1:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.321+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8fcfcc34:::obj221:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:902252c4:::obj397:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.321+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:902252c4:::obj397:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:902252c4:::obj397:head 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:902252c4:::obj397:head v 107'397 2026-02-11T23:31:17.321+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.321+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.322+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.322+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:8fcfcc34:::obj221:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8fcfcc34:::obj221:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 112/112 lb 1:8fcfcc34:::obj221:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:8fcfcc34:::obj221:head) v3 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8f7cdb2a:::obj538:head local-lis/les=109/110 n=338 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:8fcfcc34:::obj221:head) 2026-02-11T23:31:17.322+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:8fcfcc34:::obj221:head) v3 finish latency 0.000011 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.322+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.322+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa1500 con 0x3752000 2026-02-11T23:31:17.322+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:902252c4:::obj397:head)]) -- 0x4fa0c00 con 0x3752000 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.322+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 718 ==== MOSDPGPush(1.0 112/109 [PushOp(1:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x512f520 con 0x3752000 2026-02-11T23:31:17.322+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 112 2026-02-11T23:31:17.322+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.322+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.322+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 719 ==== pg_backfill(progress 1.0 e 112/112 lb 1:902252c4:::obj397:head) ==== 1062+0+0 (unknown 0 0 0) 0x511d700 con 0x3752000 2026-02-11T23:31:17.322+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:902252c4:::obj397:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.322+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.322+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:902252c4:::obj397:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.322+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.322+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.322+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:902252c4:::obj397:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.322+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.322+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.322+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.322+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:90d5bfbc:::obj593:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.322+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:90d5bfbc:::obj593:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.322+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:90d5bfbc:::obj593:head 2026-02-11T23:31:17.322+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:90d5bfbc:::obj593:head v 107'593 2026-02-11T23:31:17.322+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.322+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.322+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.322+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:902252c4:::obj397:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:902252c4:::obj397:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 112/112 lb 1:902252c4:::obj397:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:902252c4:::obj397:head) v3 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:8fcfcc34:::obj221:head local-lis/les=109/110 n=339 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:902252c4:::obj397:head) 2026-02-11T23:31:17.322+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:902252c4:::obj397:head) v3 finish latency 0.000012 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.322+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.322+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.322+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4780 con 0x3752000 2026-02-11T23:31:17.322+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:90d5bfbc:::obj593:head)]) -- 0x4fa0a80 con 0x3752000 2026-02-11T23:31:17.322+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.323+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 720 ==== MOSDPGPush(1.0 112/109 [PushOp(1:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x512fa00 con 0x3752000 2026-02-11T23:31:17.323+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 112 2026-02-11T23:31:17.323+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.323+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 721 ==== pg_backfill(progress 1.0 e 112/112 lb 1:90d5bfbc:::obj593:head) ==== 1062+0+0 (unknown 0 0 0) 0x513e000 con 0x3752000 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.323+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:90d5bfbc:::obj593:head) v3 prio 15 type 95 cost 0 latency 0.000014 epoch 112 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.323+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:90d5bfbc:::obj593:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:90d5bfbc:::obj593:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:917e3523:::obj419:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:917e3523:::obj419:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:917e3523:::obj419:head 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:917e3523:::obj419:head v 107'419 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.323+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:90d5bfbc:::obj593:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:90d5bfbc:::obj593:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 112/112 lb 1:90d5bfbc:::obj593:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:90d5bfbc:::obj593:head) v3 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:902252c4:::obj397:head local-lis/les=109/110 n=340 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:90d5bfbc:::obj593:head) 2026-02-11T23:31:17.323+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:90d5bfbc:::obj593:head) v3 finish latency 0.000012 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.323+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa1800 con 0x3752000 2026-02-11T23:31:17.323+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:917e3523:::obj419:head)]) -- 0x4fa0900 con 0x3752000 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.323+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 722 ==== MOSDPGPush(1.0 112/109 [PushOp(1:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5142000 con 0x3752000 2026-02-11T23:31:17.323+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 112 2026-02-11T23:31:17.323+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.323+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 723 ==== pg_backfill(progress 1.0 e 112/112 lb 1:917e3523:::obj419:head) ==== 1062+0+0 (unknown 0 0 0) 0x513e580 con 0x3752000 2026-02-11T23:31:17.323+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:917e3523:::obj419:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.323+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:917e3523:::obj419:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:917e3523:::obj419:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:91fc975b:::obj401:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:91fc975b:::obj401:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:91fc975b:::obj401:head 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:91fc975b:::obj401:head v 107'401 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.323+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.323+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:917e3523:::obj419:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:917e3523:::obj419:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 112/112 lb 1:917e3523:::obj419:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:917e3523:::obj419:head) v3 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:90d5bfbc:::obj593:head local-lis/les=109/110 n=341 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:917e3523:::obj419:head) 2026-02-11T23:31:17.323+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:917e3523:::obj419:head) v3 finish latency 0.000011 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.323+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.323+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.323+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.323+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd0d80 con 0x3752000 2026-02-11T23:31:17.323+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:91fc975b:::obj401:head)]) -- 0x4fa0780 con 0x3752000 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.324+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 724 ==== MOSDPGPush(1.0 112/109 [PushOp(1:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51424e0 con 0x3752000 2026-02-11T23:31:17.324+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.324+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.324+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.324+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.324+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 725 ==== pg_backfill(progress 1.0 e 112/112 lb 1:91fc975b:::obj401:head) ==== 1062+0+0 (unknown 0 0 0) 0x513eb00 con 0x3752000 2026-02-11T23:31:17.324+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.324+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.324+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:91fc975b:::obj401:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 112 2026-02-11T23:31:17.324+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:91fc975b:::obj401:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.324+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.324+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:91fc975b:::obj401:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.324+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.324+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.324+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:92b89151:::obj431:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.324+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:92b89151:::obj431:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.324+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:92b89151:::obj431:head 2026-02-11T23:31:17.324+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:92b89151:::obj431:head v 107'431 2026-02-11T23:31:17.324+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.324+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.324+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.324+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:91fc975b:::obj401:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.324+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:91fc975b:::obj401:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 112/112 lb 1:91fc975b:::obj401:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.324+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.324+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:91fc975b:::obj401:head) v3 2026-02-11T23:31:17.324+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:917e3523:::obj419:head local-lis/les=109/110 n=342 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:91fc975b:::obj401:head) 2026-02-11T23:31:17.324+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.324+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:91fc975b:::obj401:head) v3 finish latency 0.000013 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.324+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.324+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.324+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd0780 con 0x3752000 2026-02-11T23:31:17.324+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:92b89151:::obj431:head)]) -- 0x4fa0600 con 0x3752000 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.324+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 726 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51429c0 con 0x3752000 2026-02-11T23:31:17.324+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 112 2026-02-11T23:31:17.324+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.324+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 727 ==== pg_backfill(progress 1.0 e 112/112 lb 1:92b89151:::obj431:head) ==== 1062+0+0 (unknown 0 0 0) 0x513f080 con 0x3752000 2026-02-11T23:31:17.324+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:92b89151:::obj431:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.324+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:92b89151:::obj431:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.324+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.324+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 112/109 [PushOp(1:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.324+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.324+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.324+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:92b89151:::obj431:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.324+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.324+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9315b0b3:::obj555:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9315b0b3:::obj555:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9315b0b3:::obj555:head 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9315b0b3:::obj555:head v 107'555 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.325+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:92b89151:::obj431:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:92b89151:::obj431:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 1.0 e 112/112 lb 1:92b89151:::obj431:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:92b89151:::obj431:head) v3 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:91fc975b:::obj401:head local-lis/les=109/110 n=343 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:92b89151:::obj431:head) 2026-02-11T23:31:17.325+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:92b89151:::obj431:head) v3 finish latency 0.000012 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.325+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa1e00 con 0x3752000 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.325+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9315b0b3:::obj555:head)]) -- 0x4fa1b00 con 0x3752000 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.325+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 728 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5142ea0 con 0x3752000 2026-02-11T23:31:17.325+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.325+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.325+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 729 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9315b0b3:::obj555:head) ==== 1062+0+0 (unknown 0 0 0) 0x513f600 con 0x3752000 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.325+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9315b0b3:::obj555:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 112 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.325+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9315b0b3:::obj555:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 112/109 [PushOp(1:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9315b0b3:::obj555:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9397791a:::obj547:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9397791a:::obj547:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9397791a:::obj547:head 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9397791a:::obj547:head v 107'547 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.325+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.325+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9315b0b3:::obj555:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9315b0b3:::obj555:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 112/112 lb 1:9315b0b3:::obj555:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9315b0b3:::obj555:head) v3 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:92b89151:::obj431:head local-lis/les=109/110 n=344 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9315b0b3:::obj555:head) 2026-02-11T23:31:17.325+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9315b0b3:::obj555:head) v3 finish latency 0.000015 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.325+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.325+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.325+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd1680 con 0x3752000 2026-02-11T23:31:17.325+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9397791a:::obj547:head)]) -- 0x4fa0480 con 0x3752000 2026-02-11T23:31:17.325+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.326+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 730 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5143380 con 0x3752000 2026-02-11T23:31:17.326+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 112 2026-02-11T23:31:17.326+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.326+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 731 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9397791a:::obj547:head) ==== 1062+0+0 (unknown 0 0 0) 0x513fb80 con 0x3752000 2026-02-11T23:31:17.326+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9397791a:::obj547:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.326+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9397791a:::obj547:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 112/109 [PushOp(1:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9397791a:::obj547:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9455f78c:::obj228:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9455f78c:::obj228:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9455f78c:::obj228:head 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9455f78c:::obj228:head v 107'228 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.326+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9397791a:::obj547:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9397791a:::obj547:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 112/112 lb 1:9397791a:::obj547:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9397791a:::obj547:head) v3 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9315b0b3:::obj555:head local-lis/les=109/110 n=345 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9397791a:::obj547:head) 2026-02-11T23:31:17.326+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9397791a:::obj547:head) v3 finish latency 0.000012 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.326+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fea480 con 0x3752000 2026-02-11T23:31:17.326+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9455f78c:::obj228:head)]) -- 0x4fa0300 con 0x3752000 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.326+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 732 ==== MOSDPGPush(1.0 112/109 [PushOp(1:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5143860 con 0x3752000 2026-02-11T23:31:17.326+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 112 2026-02-11T23:31:17.326+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.326+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 733 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9455f78c:::obj228:head) ==== 1062+0+0 (unknown 0 0 0) 0x5140100 con 0x3752000 2026-02-11T23:31:17.326+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9455f78c:::obj228:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.326+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9455f78c:::obj228:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 112/109 [PushOp(1:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9455f78c:::obj228:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:945b6e2e:::obj114:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:945b6e2e:::obj114:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:945b6e2e:::obj114:head 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:945b6e2e:::obj114:head v 107'114 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.326+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.326+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.326+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9455f78c:::obj228:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9455f78c:::obj228:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 1.0 e 112/112 lb 1:9455f78c:::obj228:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9455f78c:::obj228:head) v3 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9397791a:::obj547:head local-lis/les=109/110 n=346 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9455f78c:::obj228:head) 2026-02-11T23:31:17.326+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.326+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9455f78c:::obj228:head) v3 finish latency 0.000014 2026-02-11T23:31:17.326+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.327+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.327+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.327+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fea180 con 0x3752000 2026-02-11T23:31:17.327+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:945b6e2e:::obj114:head)]) -- 0x4fe4600 con 0x3752000 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.327+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 734 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5143d40 con 0x3752000 2026-02-11T23:31:17.327+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 112 2026-02-11T23:31:17.327+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.327+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.327+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 735 ==== pg_backfill(progress 1.0 e 112/112 lb 1:945b6e2e:::obj114:head) ==== 1062+0+0 (unknown 0 0 0) 0x5140680 con 0x3752000 2026-02-11T23:31:17.327+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.327+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:945b6e2e:::obj114:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 112 2026-02-11T23:31:17.327+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:945b6e2e:::obj114:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.327+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.327+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.327+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 112/109 [PushOp(1:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.327+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.327+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:945b6e2e:::obj114:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.327+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.327+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9477fafa:::obj499:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.327+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9477fafa:::obj499:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.327+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9477fafa:::obj499:head 2026-02-11T23:31:17.327+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9477fafa:::obj499:head v 107'499 2026-02-11T23:31:17.327+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.327+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.327+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.327+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:945b6e2e:::obj114:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.327+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:945b6e2e:::obj114:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 112/112 lb 1:945b6e2e:::obj114:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.327+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.327+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:945b6e2e:::obj114:head) v3 2026-02-11T23:31:17.327+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9455f78c:::obj228:head local-lis/les=109/110 n=347 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:945b6e2e:::obj114:head) 2026-02-11T23:31:17.327+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.327+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:945b6e2e:::obj114:head) v3 finish latency 0.000016 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.327+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.327+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.327+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d97b00 con 0x3752000 2026-02-11T23:31:17.327+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9477fafa:::obj499:head)]) -- 0x4fa0180 con 0x3752000 2026-02-11T23:31:17.327+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.328+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 736 ==== MOSDPGPush(1.0 112/109 [PushOp(1:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5156340 con 0x3752000 2026-02-11T23:31:17.328+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.328+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.328+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 737 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9477fafa:::obj499:head) ==== 1062+0+0 (unknown 0 0 0) 0x5140c00 con 0x3752000 2026-02-11T23:31:17.328+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9477fafa:::obj499:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.328+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9477fafa:::obj499:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9477fafa:::obj499:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:95a9a074:::obj509:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:95a9a074:::obj509:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:95a9a074:::obj509:head 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:95a9a074:::obj509:head v 107'509 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.328+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9477fafa:::obj499:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9477fafa:::obj499:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 112/112 lb 1:9477fafa:::obj499:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9477fafa:::obj499:head) v3 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:945b6e2e:::obj114:head local-lis/les=109/110 n=348 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9477fafa:::obj499:head) 2026-02-11T23:31:17.328+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9477fafa:::obj499:head) v3 finish latency 0.000012 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.328+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd0180 con 0x3752000 2026-02-11T23:31:17.328+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:95a9a074:::obj509:head)]) -- 0x4dd1380 con 0x3752000 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.328+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 738 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5156820 con 0x3752000 2026-02-11T23:31:17.328+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 112 2026-02-11T23:31:17.328+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.328+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 739 ==== pg_backfill(progress 1.0 e 112/112 lb 1:95a9a074:::obj509:head) ==== 1062+0+0 (unknown 0 0 0) 0x5141180 con 0x3752000 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.328+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:95a9a074:::obj509:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.328+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:95a9a074:::obj509:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:95a9a074:::obj509:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9604ab91:::obj494:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9604ab91:::obj494:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9604ab91:::obj494:head 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9604ab91:::obj494:head v 107'494 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.328+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.328+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:95a9a074:::obj509:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:95a9a074:::obj509:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 112/112 lb 1:95a9a074:::obj509:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:95a9a074:::obj509:head) v3 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9477fafa:::obj499:head local-lis/les=109/110 n=349 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:95a9a074:::obj509:head) 2026-02-11T23:31:17.328+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:95a9a074:::obj509:head) v3 finish latency 0.000016 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.328+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.328+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.328+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.329+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d97e00 con 0x3752000 2026-02-11T23:31:17.329+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9604ab91:::obj494:head)]) -- 0x4fa0000 con 0x3752000 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.329+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 740 ==== MOSDPGPush(1.0 112/109 [PushOp(1:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5156d00 con 0x3752000 2026-02-11T23:31:17.329+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 112 2026-02-11T23:31:17.329+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.329+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.329+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 741 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9604ab91:::obj494:head) ==== 1062+0+0 (unknown 0 0 0) 0x5141700 con 0x3752000 2026-02-11T23:31:17.329+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.329+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9604ab91:::obj494:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.329+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9604ab91:::obj494:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.329+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.329+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.329+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 112/109 [PushOp(1:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.329+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9604ab91:::obj494:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.329+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.329+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.329+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:96183f21:::obj585:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.329+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:96183f21:::obj585:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.329+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:96183f21:::obj585:head 2026-02-11T23:31:17.329+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:96183f21:::obj585:head v 107'585 2026-02-11T23:31:17.329+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.329+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.329+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.329+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9604ab91:::obj494:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.329+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9604ab91:::obj494:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 112/112 lb 1:9604ab91:::obj494:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.329+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.329+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9604ab91:::obj494:head) v3 2026-02-11T23:31:17.329+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:95a9a074:::obj509:head local-lis/les=109/110 n=350 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9604ab91:::obj494:head) 2026-02-11T23:31:17.329+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.329+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9604ab91:::obj494:head) v3 finish latency 0.000013 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.329+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.329+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.329+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fb9b00 con 0x3752000 2026-02-11T23:31:17.329+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:96183f21:::obj585:head)]) -- 0x4f6fe00 con 0x3752000 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.329+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 742 ==== MOSDPGPush(1.0 112/109 [PushOp(1:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51571e0 con 0x3752000 2026-02-11T23:31:17.329+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.329+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.329+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.329+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.329+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 743 ==== pg_backfill(progress 1.0 e 112/112 lb 1:96183f21:::obj585:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162000 con 0x3752000 2026-02-11T23:31:17.329+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.329+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.329+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:96183f21:::obj585:head) v3 prio 15 type 95 cost 0 latency 0.000013 epoch 112 2026-02-11T23:31:17.329+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.329+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:96183f21:::obj585:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.329+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.329+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:96183f21:::obj585:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:96d0cad0:::obj307:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.330+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:96d0cad0:::obj307:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:96d0cad0:::obj307:head 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:96d0cad0:::obj307:head v 107'307 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.330+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.330+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:96183f21:::obj585:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:96183f21:::obj585:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 112/112 lb 1:96183f21:::obj585:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:96183f21:::obj585:head) v3 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9604ab91:::obj494:head local-lis/les=109/110 n=351 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:96183f21:::obj585:head) 2026-02-11T23:31:17.330+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:96183f21:::obj585:head) v3 finish latency 0.000014 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.330+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.330+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fb8900 con 0x3752000 2026-02-11T23:31:17.330+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:96d0cad0:::obj307:head)]) -- 0x4f6fc80 con 0x3752000 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.330+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 744 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51576c0 con 0x3752000 2026-02-11T23:31:17.330+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 112 2026-02-11T23:31:17.330+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.330+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 745 ==== pg_backfill(progress 1.0 e 112/112 lb 1:96d0cad0:::obj307:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162580 con 0x3752000 2026-02-11T23:31:17.330+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:96d0cad0:::obj307:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.330+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.330+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:96d0cad0:::obj307:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.330+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.330+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.330+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 112/109 [PushOp(1:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:96d0cad0:::obj307:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9720f72d:::obj90:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.330+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9720f72d:::obj90:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9720f72d:::obj90:head 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9720f72d:::obj90:head v 107'90 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.330+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.330+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.330+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:96d0cad0:::obj307:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:96d0cad0:::obj307:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 112/112 lb 1:96d0cad0:::obj307:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:96d0cad0:::obj307:head) v3 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96183f21:::obj585:head local-lis/les=109/110 n=352 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:96d0cad0:::obj307:head) 2026-02-11T23:31:17.330+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:96d0cad0:::obj307:head) v3 finish latency 0.000013 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.330+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.330+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.330+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa0f00 con 0x3752000 2026-02-11T23:31:17.330+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9720f72d:::obj90:head)]) -- 0x4f6fb00 con 0x3752000 2026-02-11T23:31:17.330+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.331+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 746 ==== MOSDPGPush(1.0 112/109 [PushOp(1:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5157ba0 con 0x3752000 2026-02-11T23:31:17.331+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.331+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.331+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 747 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9720f72d:::obj90:head) ==== 1061+0+0 (unknown 0 0 0) 0x5162b00 con 0x3752000 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.331+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9720f72d:::obj90:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 112 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.331+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9720f72d:::obj90:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9720f72d:::obj90:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:98019cfb:::obj381:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:98019cfb:::obj381:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:98019cfb:::obj381:head 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:98019cfb:::obj381:head v 107'381 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.331+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9720f72d:::obj90:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.331+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9720f72d:::obj90:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 112/112 lb 1:9720f72d:::obj90:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.331+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.331+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9720f72d:::obj90:head) v3 2026-02-11T23:31:17.331+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:96d0cad0:::obj307:head local-lis/les=109/110 n=353 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9720f72d:::obj90:head) 2026-02-11T23:31:17.331+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.331+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9720f72d:::obj90:head) v3 finish latency 0.000012 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.331+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.331+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd1980 con 0x3752000 2026-02-11T23:31:17.331+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:98019cfb:::obj381:head)]) -- 0x4f88180 con 0x3752000 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.331+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 748 ==== MOSDPGPush(1.0 112/109 [PushOp(1:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516c1a0 con 0x3752000 2026-02-11T23:31:17.331+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.331+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.331+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 749 ==== pg_backfill(progress 1.0 e 112/112 lb 1:98019cfb:::obj381:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163080 con 0x3752000 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.331+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:98019cfb:::obj381:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.331+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:98019cfb:::obj381:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 112/109 [PushOp(1:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:98019cfb:::obj381:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:98b68518:::obj150:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:98b68518:::obj150:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:98b68518:::obj150:head 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:98b68518:::obj150:head v 107'150 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.331+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.331+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.331+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:98019cfb:::obj381:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.331+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:98019cfb:::obj381:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 112/112 lb 1:98019cfb:::obj381:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.331+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.331+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:98019cfb:::obj381:head) v3 2026-02-11T23:31:17.331+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9720f72d:::obj90:head local-lis/les=109/110 n=354 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:98019cfb:::obj381:head) 2026-02-11T23:31:17.331+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.331+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:98019cfb:::obj381:head) v3 finish latency 0.000012 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.331+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.332+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.332+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa1080 con 0x3752000 2026-02-11T23:31:17.332+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:98b68518:::obj150:head)]) -- 0x4f53e00 con 0x3752000 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.332+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 750 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516c680 con 0x3752000 2026-02-11T23:31:17.332+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 112 2026-02-11T23:31:17.332+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.332+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 751 ==== pg_backfill(progress 1.0 e 112/112 lb 1:98b68518:::obj150:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163600 con 0x3752000 2026-02-11T23:31:17.332+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:98b68518:::obj150:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.332+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:98b68518:::obj150:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.332+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 112/109 [PushOp(1:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.332+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:98b68518:::obj150:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.332+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.332+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.332+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9956259d:::obj240:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9956259d:::obj240:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.332+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9956259d:::obj240:head 2026-02-11T23:31:17.332+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9956259d:::obj240:head v 107'240 2026-02-11T23:31:17.332+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.332+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.332+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:98b68518:::obj150:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.332+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:98b68518:::obj150:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 112/112 lb 1:98b68518:::obj150:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.332+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.332+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:98b68518:::obj150:head) v3 2026-02-11T23:31:17.332+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98019cfb:::obj381:head local-lis/les=109/110 n=355 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:98b68518:::obj150:head) 2026-02-11T23:31:17.332+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.332+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:98b68518:::obj150:head) v3 finish latency 0.000012 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.332+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.332+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4900 con 0x3752000 2026-02-11T23:31:17.332+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9956259d:::obj240:head)]) -- 0x4f6fb00 con 0x3752000 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.332+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 752 ==== MOSDPGPush(1.0 112/109 [PushOp(1:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3752000 2026-02-11T23:31:17.332+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 112 2026-02-11T23:31:17.332+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.332+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 753 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9956259d:::obj240:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163b80 con 0x3752000 2026-02-11T23:31:17.332+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9956259d:::obj240:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.332+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9956259d:::obj240:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.332+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9956259d:::obj240:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.332+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.332+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000049 MOSDPGPush(1.0 112/109 [PushOp(1:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.333+0000 7fac6dbf5640 10 monclient: _renew_subs 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:996db15f:::obj326:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.333+0000 7fac6dbf5640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:17.333+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:996db15f:::obj326:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.333+0000 7fac6dbf5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=113}) -- 0x38f5ba0 con 0x396bc00 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:996db15f:::obj326:head 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:996db15f:::obj326:head v 107'326 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.333+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.333+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9956259d:::obj240:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9956259d:::obj240:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 112/112 lb 1:9956259d:::obj240:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9956259d:::obj240:head) v3 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:98b68518:::obj150:head local-lis/les=109/110 n=356 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9956259d:::obj240:head) 2026-02-11T23:31:17.333+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9956259d:::obj240:head) v3 finish latency 0.000023 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.333+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.333+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e71c80 con 0x3752000 2026-02-11T23:31:17.333+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:996db15f:::obj326:head)]) -- 0x4f6fc80 con 0x3752000 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.333+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 754 ==== MOSDPGPush(1.0 112/109 [PushOp(1:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d040 con 0x3752000 2026-02-11T23:31:17.333+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.333+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.333+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.333+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 755 ==== pg_backfill(progress 1.0 e 112/112 lb 1:996db15f:::obj326:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3752000 2026-02-11T23:31:17.333+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:996db15f:::obj326:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.333+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.333+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:996db15f:::obj326:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.333+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.333+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:996db15f:::obj326:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:99944c24:::obj170:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.333+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:99944c24:::obj170:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:99944c24:::obj170:head 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:99944c24:::obj170:head v 107'170 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.333+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.333+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.333+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:996db15f:::obj326:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:996db15f:::obj326:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 112/112 lb 1:996db15f:::obj326:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:996db15f:::obj326:head) v3 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9956259d:::obj240:head local-lis/les=109/110 n=357 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:996db15f:::obj326:head) 2026-02-11T23:31:17.333+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:996db15f:::obj326:head) v3 finish latency 0.000012 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.333+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.333+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.333+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4dd1c80 con 0x3752000 2026-02-11T23:31:17.333+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:99944c24:::obj170:head)]) -- 0x4f6fe00 con 0x3752000 2026-02-11T23:31:17.333+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.334+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 756 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x516d520 con 0x3752000 2026-02-11T23:31:17.334+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 112 2026-02-11T23:31:17.334+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.334+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 757 ==== pg_backfill(progress 1.0 e 112/112 lb 1:99944c24:::obj170:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3752000 2026-02-11T23:31:17.334+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:99944c24:::obj170:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.334+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:99944c24:::obj170:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 112/109 [PushOp(1:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:99944c24:::obj170:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9a09113a:::obj7:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9a09113a:::obj7:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9a09113a:::obj7:head 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9a09113a:::obj7:head v 107'7 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.334+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:99944c24:::obj170:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.334+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:99944c24:::obj170:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 112/112 lb 1:99944c24:::obj170:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.334+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.334+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:99944c24:::obj170:head) v3 2026-02-11T23:31:17.334+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:996db15f:::obj326:head local-lis/les=109/110 n=358 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:99944c24:::obj170:head) 2026-02-11T23:31:17.334+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.334+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:99944c24:::obj170:head) v3 finish latency 0.000011 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.334+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.334+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4feb500 con 0x3752000 2026-02-11T23:31:17.334+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9a09113a:::obj7:head)]) -- 0x4fa0000 con 0x3752000 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.334+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 758 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516da00 con 0x3752000 2026-02-11T23:31:17.334+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 112 2026-02-11T23:31:17.334+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.334+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 759 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9a09113a:::obj7:head) ==== 1060+0+0 (unknown 0 0 0) 0x5164c00 con 0x3752000 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.334+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9a09113a:::obj7:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.334+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9a09113a:::obj7:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 112/109 [PushOp(1:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9a09113a:::obj7:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9bbc8a19:::obj348:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9bbc8a19:::obj348:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9bbc8a19:::obj348:head 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9bbc8a19:::obj348:head v 107'348 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.334+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.334+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.334+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9a09113a:::obj7:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.334+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9a09113a:::obj7:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 112/112 lb 1:9a09113a:::obj7:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.334+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.334+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9a09113a:::obj7:head) v3 2026-02-11T23:31:17.334+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:99944c24:::obj170:head local-lis/les=109/110 n=359 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9a09113a:::obj7:head) 2026-02-11T23:31:17.334+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.334+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9a09113a:::obj7:head) v3 finish latency 0.000013 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.334+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.335+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.335+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.335+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4a80 con 0x3752000 2026-02-11T23:31:17.335+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9bbc8a19:::obj348:head)]) -- 0x4d97e00 con 0x3752000 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.335+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 760 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182000 con 0x3752000 2026-02-11T23:31:17.335+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 112 2026-02-11T23:31:17.335+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.335+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 761 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9bbc8a19:::obj348:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165180 con 0x3752000 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.335+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9bbc8a19:::obj348:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.335+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9bbc8a19:::obj348:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.335+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 112/109 [PushOp(1:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.335+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.335+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.335+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9bbc8a19:::obj348:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.335+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.335+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.335+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9c3090f1:::obj522:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9c3090f1:::obj522:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.335+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9c3090f1:::obj522:head 2026-02-11T23:31:17.335+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9c3090f1:::obj522:head v 107'522 2026-02-11T23:31:17.335+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.335+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.335+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.335+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.335+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9bbc8a19:::obj348:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.335+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9bbc8a19:::obj348:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 112/112 lb 1:9bbc8a19:::obj348:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.335+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.335+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9bbc8a19:::obj348:head) v3 2026-02-11T23:31:17.335+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9a09113a:::obj7:head local-lis/les=109/110 n=360 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9bbc8a19:::obj348:head) 2026-02-11T23:31:17.335+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.335+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9bbc8a19:::obj348:head) v3 finish latency 0.000012 2026-02-11T23:31:17.335+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.335+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.335+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.335+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e71800 con 0x3752000 2026-02-11T23:31:17.335+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9c3090f1:::obj522:head)]) -- 0x4e71b00 con 0x3752000 2026-02-11T23:31:17.335+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.336+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 762 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51824e0 con 0x3752000 2026-02-11T23:31:17.336+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 112 2026-02-11T23:31:17.336+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.336+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 763 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9c3090f1:::obj522:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3752000 2026-02-11T23:31:17.336+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9c3090f1:::obj522:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.336+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9c3090f1:::obj522:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 112/109 [PushOp(1:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9c3090f1:::obj522:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9c626db4:::obj119:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9c626db4:::obj119:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9c626db4:::obj119:head 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9c626db4:::obj119:head v 107'119 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.336+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9c3090f1:::obj522:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.336+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9c3090f1:::obj522:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 112/112 lb 1:9c3090f1:::obj522:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.336+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.336+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9c3090f1:::obj522:head) v3 2026-02-11T23:31:17.336+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9bbc8a19:::obj348:head local-lis/les=109/110 n=361 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9c3090f1:::obj522:head) 2026-02-11T23:31:17.336+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.336+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9c3090f1:::obj522:head) v3 finish latency 0.000015 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.336+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.336+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5500 con 0x3752000 2026-02-11T23:31:17.336+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9c626db4:::obj119:head)]) -- 0x4dd0180 con 0x3752000 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.336+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 764 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51829c0 con 0x3752000 2026-02-11T23:31:17.336+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.336+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 112/109 [PushOp(1:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.336+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 765 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9c626db4:::obj119:head) ==== 1062+0+0 (unknown 0 0 0) 0x518e000 con 0x3752000 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.336+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9c626db4:::obj119:head) v3 prio 15 type 95 cost 0 latency 0.000022 epoch 112 2026-02-11T23:31:17.336+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9c626db4:::obj119:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9c9438f5:::obj513:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9c9438f5:::obj513:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9c9438f5:::obj513:head 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9c626db4:::obj119:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9c9438f5:::obj513:head v 107'513 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.336+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.336+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.336+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9c626db4:::obj119:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.336+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9c626db4:::obj119:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 112/112 lb 1:9c626db4:::obj119:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.336+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.336+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9c626db4:::obj119:head) v3 2026-02-11T23:31:17.336+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c3090f1:::obj522:head local-lis/les=109/110 n=362 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9c626db4:::obj119:head) 2026-02-11T23:31:17.336+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.336+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9c626db4:::obj119:head) v3 finish latency 0.000011 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.336+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.337+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.337+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e71500 con 0x3752000 2026-02-11T23:31:17.337+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9c9438f5:::obj513:head)]) -- 0x4d97b00 con 0x3752000 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.337+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 766 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182ea0 con 0x3752000 2026-02-11T23:31:17.337+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.337+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.337+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 767 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9c9438f5:::obj513:head) ==== 1062+0+0 (unknown 0 0 0) 0x518e580 con 0x3752000 2026-02-11T23:31:17.337+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9c9438f5:::obj513:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.337+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9c9438f5:::obj513:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9c9438f5:::obj513:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000050 MOSDPGPush(1.0 112/109 [PushOp(1:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9f41ce81:::obj424:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9f41ce81:::obj424:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9f41ce81:::obj424:head 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9f41ce81:::obj424:head v 107'424 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.337+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9c9438f5:::obj513:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.337+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9c9438f5:::obj513:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 112/112 lb 1:9c9438f5:::obj513:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.337+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.337+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9c9438f5:::obj513:head) v3 2026-02-11T23:31:17.337+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c626db4:::obj119:head local-lis/les=109/110 n=363 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9c9438f5:::obj513:head) 2026-02-11T23:31:17.337+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.337+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9c9438f5:::obj513:head) v3 finish latency 0.000011 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.337+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.337+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4f00 con 0x3752000 2026-02-11T23:31:17.337+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9f41ce81:::obj424:head)]) -- 0x4fea180 con 0x3752000 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.337+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 768 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183380 con 0x3752000 2026-02-11T23:31:17.337+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 112 2026-02-11T23:31:17.337+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.337+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 769 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9f41ce81:::obj424:head) ==== 1062+0+0 (unknown 0 0 0) 0x518eb00 con 0x3752000 2026-02-11T23:31:17.337+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9f41ce81:::obj424:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.337+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9f41ce81:::obj424:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.337+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.337+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9f41ce81:::obj424:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.337+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9f7e105e:::obj535:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.338+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9f7e105e:::obj535:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9f7e105e:::obj535:head 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9f7e105e:::obj535:head v 107'535 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.338+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.338+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9f41ce81:::obj424:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9f41ce81:::obj424:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 1.0 e 112/112 lb 1:9f41ce81:::obj424:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9f41ce81:::obj424:head) v3 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9c9438f5:::obj513:head local-lis/les=109/110 n=364 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9f41ce81:::obj424:head) 2026-02-11T23:31:17.338+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9f41ce81:::obj424:head) v3 finish latency 0.000011 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.338+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.338+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e71200 con 0x3752000 2026-02-11T23:31:17.338+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9f7e105e:::obj535:head)]) -- 0x4fea480 con 0x3752000 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.338+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 770 ==== MOSDPGPush(1.0 112/109 [PushOp(1:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3752000 2026-02-11T23:31:17.338+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 112 2026-02-11T23:31:17.338+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.338+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.338+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.338+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 771 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9f7e105e:::obj535:head) ==== 1062+0+0 (unknown 0 0 0) 0x518f080 con 0x3752000 2026-02-11T23:31:17.338+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.338+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9f7e105e:::obj535:head) v3 prio 15 type 95 cost 0 latency 0.000013 epoch 112 2026-02-11T23:31:17.338+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.338+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9f7e105e:::obj535:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9f7e105e:::obj535:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 112/109 [PushOp(1:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:9f8e934c:::obj456:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.338+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:9f8e934c:::obj456:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:9f8e934c:::obj456:head 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:9f8e934c:::obj456:head v 107'456 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.338+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.338+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.338+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9f7e105e:::obj535:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9f7e105e:::obj535:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 112/112 lb 1:9f7e105e:::obj535:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9f7e105e:::obj535:head) v3 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f41ce81:::obj424:head local-lis/les=109/110 n=365 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9f7e105e:::obj535:head) 2026-02-11T23:31:17.338+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9f7e105e:::obj535:head) v3 finish latency 0.000012 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.338+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.338+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.338+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5c80 con 0x3752000 2026-02-11T23:31:17.338+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:9f8e934c:::obj456:head)]) -- 0x4dd1680 con 0x3752000 2026-02-11T23:31:17.338+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.339+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 772 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183d40 con 0x3752000 2026-02-11T23:31:17.339+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 112 2026-02-11T23:31:17.339+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.339+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 773 ==== pg_backfill(progress 1.0 e 112/112 lb 1:9f8e934c:::obj456:head) ==== 1062+0+0 (unknown 0 0 0) 0x518f600 con 0x3752000 2026-02-11T23:31:17.339+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:9f8e934c:::obj456:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.339+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9f8e934c:::obj456:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 112/109 [PushOp(1:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9f8e934c:::obj456:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a0060720:::obj493:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a0060720:::obj493:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a0060720:::obj493:head 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a0060720:::obj493:head v 107'493 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.339+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:9f8e934c:::obj456:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.339+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9f8e934c:::obj456:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 112/112 lb 1:9f8e934c:::obj456:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.339+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.339+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:9f8e934c:::obj456:head) v3 2026-02-11T23:31:17.339+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f7e105e:::obj535:head local-lis/les=109/110 n=366 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:9f8e934c:::obj456:head) 2026-02-11T23:31:17.339+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.339+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:9f8e934c:::obj456:head) v3 finish latency 0.000012 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.339+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.339+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e71080 con 0x3752000 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.339+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a0060720:::obj493:head)]) -- 0x4fa1e00 con 0x3752000 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.339+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 774 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2340 con 0x3752000 2026-02-11T23:31:17.339+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 112 2026-02-11T23:31:17.339+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.339+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 775 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a0060720:::obj493:head) ==== 1062+0+0 (unknown 0 0 0) 0x518fb80 con 0x3752000 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.339+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a0060720:::obj493:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.339+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a0060720:::obj493:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a0060720:::obj493:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a0217245:::obj31:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a0217245:::obj31:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a0217245:::obj31:head 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a0217245:::obj31:head v 107'31 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.339+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.339+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.339+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a0060720:::obj493:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.339+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a0060720:::obj493:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 112/112 lb 1:a0060720:::obj493:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.339+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.339+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a0060720:::obj493:head) v3 2026-02-11T23:31:17.339+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:9f8e934c:::obj456:head local-lis/les=109/110 n=367 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a0060720:::obj493:head) 2026-02-11T23:31:17.339+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.339+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a0060720:::obj493:head) v3 finish latency 0.000011 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.339+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.340+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.340+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.340+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5800 con 0x3752000 2026-02-11T23:31:17.340+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a0217245:::obj31:head)]) -- 0x4dd0780 con 0x3752000 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.340+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 776 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3752000 2026-02-11T23:31:17.340+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 112 2026-02-11T23:31:17.340+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.340+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 777 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a0217245:::obj31:head) ==== 1061+0+0 (unknown 0 0 0) 0x5190100 con 0x3752000 2026-02-11T23:31:17.340+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a0217245:::obj31:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.340+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.340+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a0217245:::obj31:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.340+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.340+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.340+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.340+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.340+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a0217245:::obj31:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.340+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.340+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.340+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a0e9da24:::obj323:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.340+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a0e9da24:::obj323:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.340+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a0e9da24:::obj323:head 2026-02-11T23:31:17.340+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a0e9da24:::obj323:head v 107'323 2026-02-11T23:31:17.340+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.340+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.340+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.340+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a0217245:::obj31:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.340+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a0217245:::obj31:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 112/112 lb 1:a0217245:::obj31:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.340+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.340+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a0217245:::obj31:head) v3 2026-02-11T23:31:17.340+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0060720:::obj493:head local-lis/les=109/110 n=368 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a0217245:::obj31:head) 2026-02-11T23:31:17.340+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.340+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a0217245:::obj31:head) v3 finish latency 0.000011 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.340+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.340+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.340+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e70d80 con 0x3752000 2026-02-11T23:31:17.340+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a0e9da24:::obj323:head)]) -- 0x4fa0780 con 0x3752000 2026-02-11T23:31:17.340+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.341+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 778 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3752000 2026-02-11T23:31:17.341+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.341+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.341+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 779 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a0e9da24:::obj323:head) ==== 1062+0+0 (unknown 0 0 0) 0x5190680 con 0x3752000 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.341+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a0e9da24:::obj323:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 112 2026-02-11T23:31:17.341+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a0e9da24:::obj323:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a0e9da24:::obj323:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a10cad2d:::obj340:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a10cad2d:::obj340:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a10cad2d:::obj340:head 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a10cad2d:::obj340:head v 107'340 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.341+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a0e9da24:::obj323:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a0e9da24:::obj323:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 112/112 lb 1:a0e9da24:::obj323:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a0e9da24:::obj323:head) v3 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0217245:::obj31:head local-lis/les=109/110 n=369 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a0e9da24:::obj323:head) 2026-02-11T23:31:17.341+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a0e9da24:::obj323:head) v3 finish latency 0.000013 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.341+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5b00 con 0x3752000 2026-02-11T23:31:17.341+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a10cad2d:::obj340:head)]) -- 0x4fa0900 con 0x3752000 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.341+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 780 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a31e0 con 0x3752000 2026-02-11T23:31:17.341+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 112 2026-02-11T23:31:17.341+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.341+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 781 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a10cad2d:::obj340:head) ==== 1062+0+0 (unknown 0 0 0) 0x5190c00 con 0x3752000 2026-02-11T23:31:17.341+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a10cad2d:::obj340:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.341+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a10cad2d:::obj340:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 112/109 [PushOp(1:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a10cad2d:::obj340:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a16a9a6a:::obj427:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a16a9a6a:::obj427:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a16a9a6a:::obj427:head 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a16a9a6a:::obj427:head v 107'427 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.341+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a10cad2d:::obj340:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a10cad2d:::obj340:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 112/112 lb 1:a10cad2d:::obj340:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a10cad2d:::obj340:head) v3 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a0e9da24:::obj323:head local-lis/les=109/110 n=370 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a10cad2d:::obj340:head) 2026-02-11T23:31:17.341+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.341+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a10cad2d:::obj340:head) v3 finish latency 0.000013 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.341+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.341+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.341+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e70780 con 0x3752000 2026-02-11T23:31:17.341+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a16a9a6a:::obj427:head)]) -- 0x4e70a80 con 0x3752000 2026-02-11T23:31:17.341+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.342+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 782 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3752000 2026-02-11T23:31:17.342+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 112 2026-02-11T23:31:17.342+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.342+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 783 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a16a9a6a:::obj427:head) ==== 1062+0+0 (unknown 0 0 0) 0x5191180 con 0x3752000 2026-02-11T23:31:17.342+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a16a9a6a:::obj427:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.342+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a16a9a6a:::obj427:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a16a9a6a:::obj427:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000054 MOSDPGPush(1.0 112/109 [PushOp(1:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a16dc95c:::obj269:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a16dc95c:::obj269:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a16dc95c:::obj269:head 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a16dc95c:::obj269:head v 107'269 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.342+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a16a9a6a:::obj427:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.342+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a16a9a6a:::obj427:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 112/112 lb 1:a16a9a6a:::obj427:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.342+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.342+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a16a9a6a:::obj427:head) v3 2026-02-11T23:31:17.342+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a10cad2d:::obj340:head local-lis/les=109/110 n=371 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a16a9a6a:::obj427:head) 2026-02-11T23:31:17.342+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.342+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a16a9a6a:::obj427:head) v3 finish latency 0.000014 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.342+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.342+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96a80 con 0x3752000 2026-02-11T23:31:17.342+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a16dc95c:::obj269:head)]) -- 0x4fa0a80 con 0x3752000 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.342+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 784 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3ba0 con 0x3752000 2026-02-11T23:31:17.342+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 112 2026-02-11T23:31:17.342+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.342+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 785 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a16dc95c:::obj269:head) ==== 1062+0+0 (unknown 0 0 0) 0x5191700 con 0x3752000 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.342+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a16dc95c:::obj269:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.342+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a16dc95c:::obj269:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a16dc95c:::obj269:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a22120c4:::obj223:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a22120c4:::obj223:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a22120c4:::obj223:head 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a22120c4:::obj223:head v 107'223 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.342+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.342+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.342+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a16dc95c:::obj269:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.342+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a16dc95c:::obj269:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 112/112 lb 1:a16dc95c:::obj269:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.342+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.342+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a16dc95c:::obj269:head) v3 2026-02-11T23:31:17.342+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16a9a6a:::obj427:head local-lis/les=109/110 n=372 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a16dc95c:::obj269:head) 2026-02-11T23:31:17.342+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.342+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a16dc95c:::obj269:head) v3 finish latency 0.000012 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.342+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.343+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.343+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.343+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4c00 con 0x3752000 2026-02-11T23:31:17.343+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a22120c4:::obj223:head)]) -- 0x4fa0c00 con 0x3752000 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.343+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 786 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183d40 con 0x3752000 2026-02-11T23:31:17.343+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.343+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.343+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.343+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 787 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a22120c4:::obj223:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbec00 con 0x3752000 2026-02-11T23:31:17.343+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.343+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a22120c4:::obj223:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 112 2026-02-11T23:31:17.343+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.343+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.343+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a22120c4:::obj223:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a22120c4:::obj223:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.343+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.343+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.343+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.343+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.343+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a25cd2bd:::obj42:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.343+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a25cd2bd:::obj42:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.343+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a25cd2bd:::obj42:head 2026-02-11T23:31:17.343+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a25cd2bd:::obj42:head v 107'42 2026-02-11T23:31:17.343+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.343+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.343+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.343+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a22120c4:::obj223:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.343+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a22120c4:::obj223:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 112/112 lb 1:a22120c4:::obj223:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.343+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.343+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a22120c4:::obj223:head) v3 2026-02-11T23:31:17.343+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a16dc95c:::obj269:head local-lis/les=109/110 n=373 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a22120c4:::obj223:head) 2026-02-11T23:31:17.343+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.343+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a22120c4:::obj223:head) v3 finish latency 0.000011 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.343+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.343+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.343+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96d80 con 0x3752000 2026-02-11T23:31:17.343+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a25cd2bd:::obj42:head)]) -- 0x4fa0d80 con 0x3752000 2026-02-11T23:31:17.343+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.344+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 788 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3752000 2026-02-11T23:31:17.344+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.344+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.344+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 789 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a25cd2bd:::obj42:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3752000 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.344+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a25cd2bd:::obj42:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 112 2026-02-11T23:31:17.344+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a25cd2bd:::obj42:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a25cd2bd:::obj42:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a2bd80aa:::obj577:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a2bd80aa:::obj577:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a2bd80aa:::obj577:head 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a2bd80aa:::obj577:head v 107'577 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.344+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a25cd2bd:::obj42:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a25cd2bd:::obj42:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 112/112 lb 1:a25cd2bd:::obj42:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a25cd2bd:::obj42:head) v3 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a22120c4:::obj223:head local-lis/les=109/110 n=374 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a25cd2bd:::obj42:head) 2026-02-11T23:31:17.344+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a25cd2bd:::obj42:head) v3 finish latency 0.000012 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.344+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e70480 con 0x3752000 2026-02-11T23:31:17.344+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a2bd80aa:::obj577:head)]) -- 0x4fe5b00 con 0x3752000 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.344+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 790 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183380 con 0x3752000 2026-02-11T23:31:17.344+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.344+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.344+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 791 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a2bd80aa:::obj577:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3752000 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.344+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a2bd80aa:::obj577:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.344+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a2bd80aa:::obj577:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a2bd80aa:::obj577:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a3278361:::obj425:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a3278361:::obj425:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a3278361:::obj425:head 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a3278361:::obj425:head v 107'425 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.344+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.344+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a2bd80aa:::obj577:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a2bd80aa:::obj577:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 112/112 lb 1:a2bd80aa:::obj577:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a2bd80aa:::obj577:head) v3 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a25cd2bd:::obj42:head local-lis/les=109/110 n=375 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a2bd80aa:::obj577:head) 2026-02-11T23:31:17.344+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a2bd80aa:::obj577:head) v3 finish latency 0.000013 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.344+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.344+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.344+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96180 con 0x3752000 2026-02-11T23:31:17.344+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a3278361:::obj425:head)]) -- 0x4e70d80 con 0x3752000 2026-02-11T23:31:17.344+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.345+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 792 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182ea0 con 0x3752000 2026-02-11T23:31:17.345+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.345+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.345+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 793 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a3278361:::obj425:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3752000 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.345+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a3278361:::obj425:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.345+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a3278361:::obj425:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.345+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 112/109 [PushOp(1:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.345+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a3278361:::obj425:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a3411a0c:::obj160:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a3411a0c:::obj160:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a3411a0c:::obj160:head 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a3411a0c:::obj160:head v 107'160 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.345+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.345+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.345+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a3278361:::obj425:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.345+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a3278361:::obj425:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 112/112 lb 1:a3278361:::obj425:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.345+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.345+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a3278361:::obj425:head) v3 2026-02-11T23:31:17.345+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a2bd80aa:::obj577:head local-lis/les=109/110 n=376 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a3278361:::obj425:head) 2026-02-11T23:31:17.345+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.345+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a3278361:::obj425:head) v3 finish latency 0.000013 2026-02-11T23:31:17.345+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.345+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.345+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e70180 con 0x3752000 2026-02-11T23:31:17.345+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a3411a0c:::obj160:head)]) -- 0x4fe5e00 con 0x3752000 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.345+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 794 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51829c0 con 0x3752000 2026-02-11T23:31:17.345+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.345+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.345+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 795 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a3411a0c:::obj160:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3752000 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.345+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a3411a0c:::obj160:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.345+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a3411a0c:::obj160:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.345+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.345+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.345+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.345+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a3411a0c:::obj160:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.345+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a37ff878:::obj107:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a37ff878:::obj107:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a37ff878:::obj107:head 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a37ff878:::obj107:head v 107'107 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.346+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.346+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.346+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a3411a0c:::obj160:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.346+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a3411a0c:::obj160:head) v3 prio 15 cost 0 latency 0.000106 pg_backfill(progress 1.0 e 112/112 lb 1:a3411a0c:::obj160:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.346+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.346+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a3411a0c:::obj160:head) v3 2026-02-11T23:31:17.346+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3278361:::obj425:head local-lis/les=109/110 n=377 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a3411a0c:::obj160:head) 2026-02-11T23:31:17.346+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.346+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a3411a0c:::obj160:head) v3 finish latency 0.000015 2026-02-11T23:31:17.346+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.346+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.346+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6f680 con 0x3752000 2026-02-11T23:31:17.346+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a37ff878:::obj107:head)]) -- 0x4f6f800 con 0x3752000 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.346+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 796 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51824e0 con 0x3752000 2026-02-11T23:31:17.346+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.346+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.346+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 797 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a37ff878:::obj107:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3752000 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.346+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a37ff878:::obj107:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.346+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a37ff878:::obj107:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 112/109 [PushOp(1:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.346+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.346+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a37ff878:::obj107:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a383681b:::obj41:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a383681b:::obj41:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a383681b:::obj41:head 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a383681b:::obj41:head v 107'41 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.346+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.346+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.346+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a37ff878:::obj107:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.346+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a37ff878:::obj107:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 112/112 lb 1:a37ff878:::obj107:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.346+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.346+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a37ff878:::obj107:head) v3 2026-02-11T23:31:17.346+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3411a0c:::obj160:head local-lis/les=109/110 n=378 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a37ff878:::obj107:head) 2026-02-11T23:31:17.346+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.346+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a37ff878:::obj107:head) v3 finish latency 0.000014 2026-02-11T23:31:17.346+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.346+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.346+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.346+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6f500 con 0x3752000 2026-02-11T23:31:17.346+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a383681b:::obj41:head)]) -- 0x4f6f200 con 0x3752000 2026-02-11T23:31:17.346+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.347+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 798 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182000 con 0x3752000 2026-02-11T23:31:17.347+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.347+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.347+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 799 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a383681b:::obj41:head) ==== 1061+0+0 (unknown 0 0 0) 0x4decb00 con 0x3752000 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.347+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a383681b:::obj41:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.347+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a383681b:::obj41:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a383681b:::obj41:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a39a40f0:::obj561:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a39a40f0:::obj561:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a39a40f0:::obj561:head 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a39a40f0:::obj561:head v 107'561 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.347+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a383681b:::obj41:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a383681b:::obj41:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 112/112 lb 1:a383681b:::obj41:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a383681b:::obj41:head) v3 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a37ff878:::obj107:head local-lis/les=109/110 n=379 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a383681b:::obj41:head) 2026-02-11T23:31:17.347+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a383681b:::obj41:head) v3 finish latency 0.000017 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.347+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6ed80 con 0x3752000 2026-02-11T23:31:17.347+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a39a40f0:::obj561:head)]) -- 0x4fe5800 con 0x3752000 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.347+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 800 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516da00 con 0x3752000 2026-02-11T23:31:17.347+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.347+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.347+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 801 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a39a40f0:::obj561:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded080 con 0x3752000 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.347+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a39a40f0:::obj561:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.347+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a39a40f0:::obj561:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a39a40f0:::obj561:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a3cf8cd5:::obj363:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a3cf8cd5:::obj363:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a3cf8cd5:::obj363:head 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a3cf8cd5:::obj363:head v 107'363 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.347+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.347+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.347+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a39a40f0:::obj561:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a39a40f0:::obj561:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 112/112 lb 1:a39a40f0:::obj561:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a39a40f0:::obj561:head) v3 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a383681b:::obj41:head local-lis/les=109/110 n=380 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a39a40f0:::obj561:head) 2026-02-11T23:31:17.347+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.347+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a39a40f0:::obj561:head) v3 finish latency 0.000014 2026-02-11T23:31:17.347+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.348+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.348+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.348+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6ec00 con 0x3752000 2026-02-11T23:31:17.348+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a3cf8cd5:::obj363:head)]) -- 0x4f6f080 con 0x3752000 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.348+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 802 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3752000 2026-02-11T23:31:17.348+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.348+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.348+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 803 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a3cf8cd5:::obj363:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3752000 2026-02-11T23:31:17.348+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a3cf8cd5:::obj363:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.348+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a3cf8cd5:::obj363:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.348+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.348+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.348+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.348+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.348+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.348+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a3cf8cd5:::obj363:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.348+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.348+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a42b336e:::obj435:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a42b336e:::obj435:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.348+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a42b336e:::obj435:head 2026-02-11T23:31:17.348+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a42b336e:::obj435:head v 107'435 2026-02-11T23:31:17.348+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.348+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.348+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.348+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.348+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a3cf8cd5:::obj363:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.348+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a3cf8cd5:::obj363:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 112/112 lb 1:a3cf8cd5:::obj363:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.348+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.348+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a3cf8cd5:::obj363:head) v3 2026-02-11T23:31:17.348+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a39a40f0:::obj561:head local-lis/les=109/110 n=381 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a3cf8cd5:::obj363:head) 2026-02-11T23:31:17.348+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.348+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a3cf8cd5:::obj363:head) v3 finish latency 0.000017 2026-02-11T23:31:17.348+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.348+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.348+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.348+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6e300 con 0x3752000 2026-02-11T23:31:17.348+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a42b336e:::obj435:head)]) -- 0x4f53800 con 0x3752000 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.348+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 804 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d040 con 0x3752000 2026-02-11T23:31:17.348+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.348+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.348+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 805 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a42b336e:::obj435:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3752000 2026-02-11T23:31:17.348+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a42b336e:::obj435:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.348+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a42b336e:::obj435:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.348+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 112/109 [PushOp(1:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a4383432:::obj559:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a42b336e:::obj435:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a4383432:::obj559:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a4383432:::obj559:head 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a4383432:::obj559:head v 107'559 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.349+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a42b336e:::obj435:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.349+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a42b336e:::obj435:head) v3 prio 15 cost 0 latency 0.000107 pg_backfill(progress 1.0 e 112/112 lb 1:a42b336e:::obj435:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.349+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.349+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a42b336e:::obj435:head) v3 2026-02-11T23:31:17.349+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a3cf8cd5:::obj363:head local-lis/les=109/110 n=382 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a42b336e:::obj435:head) 2026-02-11T23:31:17.349+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.349+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a42b336e:::obj435:head) v3 finish latency 0.000013 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.349+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6e180 con 0x3752000 2026-02-11T23:31:17.349+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a4383432:::obj559:head)]) -- 0x4f6e600 con 0x3752000 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.349+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 806 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3752000 2026-02-11T23:31:17.349+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.349+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.349+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 807 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a4383432:::obj559:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3752000 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.349+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a4383432:::obj559:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.349+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a4383432:::obj559:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 112/109 [PushOp(1:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a4383432:::obj559:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a4518626:::obj581:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a4518626:::obj581:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a4518626:::obj581:head 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a4518626:::obj581:head v 107'581 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.349+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a4383432:::obj559:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.349+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a4383432:::obj559:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 112/112 lb 1:a4383432:::obj559:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.349+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.349+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a4383432:::obj559:head) v3 2026-02-11T23:31:17.349+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a42b336e:::obj435:head local-lis/les=109/110 n=383 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a4383432:::obj559:head) 2026-02-11T23:31:17.349+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.349+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a4383432:::obj559:head) v3 finish latency 0.000012 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.349+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.349+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.349+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f53200 con 0x3752000 2026-02-11T23:31:17.349+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a4518626:::obj581:head)]) -- 0x4f51c80 con 0x3752000 2026-02-11T23:31:17.349+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.350+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 808 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182d00 con 0x3752000 2026-02-11T23:31:17.350+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.350+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.350+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 809 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a4518626:::obj581:head) ==== 1062+0+0 (unknown 0 0 0) 0x4deec00 con 0x3752000 2026-02-11T23:31:17.350+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a4518626:::obj581:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.350+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a4518626:::obj581:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 112/109 [PushOp(1:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a4518626:::obj581:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a4664b16:::obj367:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a4664b16:::obj367:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a4664b16:::obj367:head 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a4664b16:::obj367:head v 107'367 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.350+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a4518626:::obj581:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.350+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a4518626:::obj581:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 112/112 lb 1:a4518626:::obj581:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.350+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.350+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a4518626:::obj581:head) v3 2026-02-11T23:31:17.350+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4383432:::obj559:head local-lis/les=109/110 n=384 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a4518626:::obj581:head) 2026-02-11T23:31:17.350+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.350+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a4518626:::obj581:head) v3 finish latency 0.000014 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.350+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.350+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f51b00 con 0x3752000 2026-02-11T23:31:17.350+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a4664b16:::obj367:head)]) -- 0x4e71080 con 0x3752000 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.350+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 810 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51831e0 con 0x3752000 2026-02-11T23:31:17.350+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.350+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.350+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 811 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a4664b16:::obj367:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def180 con 0x3752000 2026-02-11T23:31:17.350+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a4664b16:::obj367:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.350+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a4664b16:::obj367:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 112/109 [PushOp(1:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a4664b16:::obj367:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a52fdb40:::obj548:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a52fdb40:::obj548:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a52fdb40:::obj548:head 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a52fdb40:::obj548:head v 107'548 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.350+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.350+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.350+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.350+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a4664b16:::obj367:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.350+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a4664b16:::obj367:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 112/112 lb 1:a4664b16:::obj367:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.350+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.350+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a4664b16:::obj367:head) v3 2026-02-11T23:31:17.350+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4518626:::obj581:head local-lis/les=109/110 n=385 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a4664b16:::obj367:head) 2026-02-11T23:31:17.350+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a4664b16:::obj367:head) v3 finish latency 0.000017 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.351+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.351+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f52f00 con 0x3752000 2026-02-11T23:31:17.351+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a52fdb40:::obj548:head)]) -- 0x4fe5c80 con 0x3752000 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.351+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 812 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3752000 2026-02-11T23:31:17.351+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.351+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.351+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 813 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a52fdb40:::obj548:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def700 con 0x3752000 2026-02-11T23:31:17.351+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a52fdb40:::obj548:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.351+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a52fdb40:::obj548:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 112/109 [PushOp(1:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.351+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.351+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a52fdb40:::obj548:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a53b2263:::obj137:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a53b2263:::obj137:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a53b2263:::obj137:head 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a53b2263:::obj137:head v 107'137 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.351+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.351+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.351+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a52fdb40:::obj548:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.351+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a52fdb40:::obj548:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 112/112 lb 1:a52fdb40:::obj548:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.351+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.351+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a52fdb40:::obj548:head) v3 2026-02-11T23:31:17.351+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a4664b16:::obj367:head local-lis/les=109/110 n=386 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a52fdb40:::obj548:head) 2026-02-11T23:31:17.351+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.351+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a52fdb40:::obj548:head) v3 finish latency 0.000013 2026-02-11T23:31:17.351+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.351+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.351+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.351+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f51800 con 0x3752000 2026-02-11T23:31:17.351+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a53b2263:::obj137:head)]) -- 0x4f51980 con 0x3752000 2026-02-11T23:31:17.351+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.352+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 814 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3752000 2026-02-11T23:31:17.352+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.352+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.352+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 815 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a53b2263:::obj137:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3752000 2026-02-11T23:31:17.352+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a53b2263:::obj137:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.352+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a53b2263:::obj137:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a53b2263:::obj137:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a555be2b:::obj75:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a555be2b:::obj75:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a555be2b:::obj75:head 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a555be2b:::obj75:head v 107'75 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.352+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a53b2263:::obj137:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a53b2263:::obj137:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 112/112 lb 1:a53b2263:::obj137:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a53b2263:::obj137:head) v3 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a52fdb40:::obj548:head local-lis/les=109/110 n=387 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a53b2263:::obj137:head) 2026-02-11T23:31:17.352+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a53b2263:::obj137:head) v3 finish latency 0.000010 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.352+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f52a80 con 0x3752000 2026-02-11T23:31:17.352+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a555be2b:::obj75:head)]) -- 0x4e71200 con 0x3752000 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.352+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 816 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3752000 2026-02-11T23:31:17.352+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.352+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.352+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 817 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a555be2b:::obj75:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3752000 2026-02-11T23:31:17.352+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a555be2b:::obj75:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.352+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a555be2b:::obj75:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 112/109 [PushOp(1:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a555be2b:::obj75:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a557efb1:::obj8:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a557efb1:::obj8:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a557efb1:::obj8:head 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a557efb1:::obj8:head v 107'8 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.352+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.352+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a555be2b:::obj75:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a555be2b:::obj75:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 112/112 lb 1:a555be2b:::obj75:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a555be2b:::obj75:head) v3 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a53b2263:::obj137:head local-lis/les=109/110 n=388 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a555be2b:::obj75:head) 2026-02-11T23:31:17.352+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a555be2b:::obj75:head) v3 finish latency 0.000013 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.352+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.352+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.352+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f51680 con 0x3752000 2026-02-11T23:31:17.352+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a557efb1:::obj8:head)]) -- 0x4fea180 con 0x3752000 2026-02-11T23:31:17.352+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.353+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 818 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3752000 2026-02-11T23:31:17.353+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.353+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.353+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 819 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a557efb1:::obj8:head) ==== 1060+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3752000 2026-02-11T23:31:17.353+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.353+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a557efb1:::obj8:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.353+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a557efb1:::obj8:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.353+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.353+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.353+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.353+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.353+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a557efb1:::obj8:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.353+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.353+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.353+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a5c2f6e2:::obj111:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.353+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a5c2f6e2:::obj111:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.353+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a5c2f6e2:::obj111:head 2026-02-11T23:31:17.353+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a5c2f6e2:::obj111:head v 107'111 2026-02-11T23:31:17.353+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.353+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.353+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.353+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a557efb1:::obj8:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.353+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a557efb1:::obj8:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 112/112 lb 1:a557efb1:::obj8:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.353+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.353+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a557efb1:::obj8:head) v3 2026-02-11T23:31:17.353+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a555be2b:::obj75:head local-lis/les=109/110 n=389 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a557efb1:::obj8:head) 2026-02-11T23:31:17.353+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.353+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a557efb1:::obj8:head) v3 finish latency 0.000013 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.353+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.353+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.353+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f52780 con 0x3752000 2026-02-11T23:31:17.353+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a5c2f6e2:::obj111:head)]) -- 0x4fe4f00 con 0x3752000 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.353+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 820 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3380 con 0x3752000 2026-02-11T23:31:17.353+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.353+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.353+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 821 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a5c2f6e2:::obj111:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d080 con 0x3752000 2026-02-11T23:31:17.353+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a5c2f6e2:::obj111:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.353+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a5c2f6e2:::obj111:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.353+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.353+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 112/109 [PushOp(1:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a5c2f6e2:::obj111:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a5ea1435:::obj539:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a5ea1435:::obj539:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a5ea1435:::obj539:head 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a5ea1435:::obj539:head v 107'539 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.354+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000127 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a5c2f6e2:::obj111:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a5c2f6e2:::obj111:head) v3 prio 15 cost 0 latency 0.000163 pg_backfill(progress 1.0 e 112/112 lb 1:a5c2f6e2:::obj111:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a5c2f6e2:::obj111:head) v3 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a557efb1:::obj8:head local-lis/les=109/110 n=390 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a5c2f6e2:::obj111:head) 2026-02-11T23:31:17.354+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a5c2f6e2:::obj111:head) v3 finish latency 0.000015 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.354+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f51380 con 0x3752000 2026-02-11T23:31:17.354+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a5ea1435:::obj539:head)]) -- 0x4f51500 con 0x3752000 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.354+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 822 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3752000 2026-02-11T23:31:17.354+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.354+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.354+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 823 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a5ea1435:::obj539:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3752000 2026-02-11T23:31:17.354+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a5ea1435:::obj539:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.354+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a5ea1435:::obj539:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a5ea1435:::obj539:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(1.0 112/109 [PushOp(1:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a601bbe9:::obj562:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a601bbe9:::obj562:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a601bbe9:::obj562:head 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a601bbe9:::obj562:head v 107'562 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.354+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.354+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a5ea1435:::obj539:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a5ea1435:::obj539:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 112/112 lb 1:a5ea1435:::obj539:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a5ea1435:::obj539:head) v3 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5c2f6e2:::obj111:head local-lis/les=109/110 n=391 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a5ea1435:::obj539:head) 2026-02-11T23:31:17.354+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a5ea1435:::obj539:head) v3 finish latency 0.000021 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.354+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.354+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.354+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f52480 con 0x3752000 2026-02-11T23:31:17.354+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a601bbe9:::obj562:head)]) -- 0x4e71500 con 0x3752000 2026-02-11T23:31:17.354+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.355+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 824 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3752000 2026-02-11T23:31:17.355+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.355+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.355+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 825 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a601bbe9:::obj562:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3752000 2026-02-11T23:31:17.355+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a601bbe9:::obj562:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.355+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a601bbe9:::obj562:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a601bbe9:::obj562:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a61ad63e:::obj13:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a61ad63e:::obj13:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a61ad63e:::obj13:head 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a61ad63e:::obj13:head v 107'13 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.355+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a601bbe9:::obj562:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.355+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a601bbe9:::obj562:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 112/112 lb 1:a601bbe9:::obj562:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.355+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.355+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a601bbe9:::obj562:head) v3 2026-02-11T23:31:17.355+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a5ea1435:::obj539:head local-lis/les=109/110 n=392 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a601bbe9:::obj562:head) 2026-02-11T23:31:17.355+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.355+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a601bbe9:::obj562:head) v3 finish latency 0.000014 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.355+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.355+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f50f00 con 0x3752000 2026-02-11T23:31:17.355+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a61ad63e:::obj13:head)]) -- 0x4fe5500 con 0x3752000 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.355+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 826 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3752000 2026-02-11T23:31:17.355+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.355+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.355+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 827 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a61ad63e:::obj13:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3752000 2026-02-11T23:31:17.355+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a61ad63e:::obj13:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.355+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a61ad63e:::obj13:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a61ad63e:::obj13:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a630f391:::obj375:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a630f391:::obj375:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a630f391:::obj375:head 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a630f391:::obj375:head v 107'375 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.355+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.355+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.355+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.355+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a61ad63e:::obj13:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.355+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a61ad63e:::obj13:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 112/112 lb 1:a61ad63e:::obj13:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.355+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.355+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a61ad63e:::obj13:head) v3 2026-02-11T23:31:17.355+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a601bbe9:::obj562:head local-lis/les=109/110 n=393 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a61ad63e:::obj13:head) 2026-02-11T23:31:17.355+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.355+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a61ad63e:::obj13:head) v3 finish latency 0.000012 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.356+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.356+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.356+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f50a80 con 0x3752000 2026-02-11T23:31:17.356+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a630f391:::obj375:head)]) -- 0x4e71b00 con 0x3752000 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.356+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 828 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3752000 2026-02-11T23:31:17.356+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.356+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.356+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.356+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 829 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a630f391:::obj375:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3752000 2026-02-11T23:31:17.356+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.356+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a630f391:::obj375:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.356+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a630f391:::obj375:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.356+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.356+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.356+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.356+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a630f391:::obj375:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.356+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.356+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.356+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a6b27c5f:::obj420:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.356+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a6b27c5f:::obj420:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.356+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a6b27c5f:::obj420:head 2026-02-11T23:31:17.356+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a6b27c5f:::obj420:head v 107'420 2026-02-11T23:31:17.356+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.356+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.356+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.356+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a630f391:::obj375:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.356+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a630f391:::obj375:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 112/112 lb 1:a630f391:::obj375:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.356+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.356+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a630f391:::obj375:head) v3 2026-02-11T23:31:17.356+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a61ad63e:::obj13:head local-lis/les=109/110 n=394 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a630f391:::obj375:head) 2026-02-11T23:31:17.356+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.356+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a630f391:::obj375:head) v3 finish latency 0.000013 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.356+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.356+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.356+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f52f00 con 0x3752000 2026-02-11T23:31:17.356+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a6b27c5f:::obj420:head)]) -- 0x4d97e00 con 0x3752000 2026-02-11T23:31:17.356+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.357+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 830 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3752000 2026-02-11T23:31:17.357+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.357+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.357+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 831 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a6b27c5f:::obj420:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f180 con 0x3752000 2026-02-11T23:31:17.357+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a6b27c5f:::obj420:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.357+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a6b27c5f:::obj420:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 112/109 [PushOp(1:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a6b27c5f:::obj420:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a6f174aa:::obj203:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a6f174aa:::obj203:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a6f174aa:::obj203:head 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a6f174aa:::obj203:head v 107'203 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.357+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a6b27c5f:::obj420:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a6b27c5f:::obj420:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 112/112 lb 1:a6b27c5f:::obj420:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a6b27c5f:::obj420:head) v3 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a630f391:::obj375:head local-lis/les=109/110 n=395 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a6b27c5f:::obj420:head) 2026-02-11T23:31:17.357+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a6b27c5f:::obj420:head) v3 finish latency 0.000011 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.357+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e71080 con 0x3752000 2026-02-11T23:31:17.357+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a6f174aa:::obj203:head)]) -- 0x4fa0000 con 0x3752000 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.357+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 832 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183520 con 0x3752000 2026-02-11T23:31:17.357+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.357+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.357+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 833 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a6f174aa:::obj203:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3752000 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.357+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a6f174aa:::obj203:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.357+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a6f174aa:::obj203:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a6f174aa:::obj203:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a8709cbe:::obj458:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a8709cbe:::obj458:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a8709cbe:::obj458:head 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a8709cbe:::obj458:head v 107'458 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.357+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.357+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a6f174aa:::obj203:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a6f174aa:::obj203:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 112/112 lb 1:a6f174aa:::obj203:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a6f174aa:::obj203:head) v3 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6b27c5f:::obj420:head local-lis/les=109/110 n=396 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a6f174aa:::obj203:head) 2026-02-11T23:31:17.357+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a6f174aa:::obj203:head) v3 finish latency 0.000011 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.357+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.357+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.357+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f51b00 con 0x3752000 2026-02-11T23:31:17.357+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a8709cbe:::obj458:head)]) -- 0x4f6fe00 con 0x3752000 2026-02-11T23:31:17.357+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.358+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 834 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183040 con 0x3752000 2026-02-11T23:31:17.358+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.358+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.358+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 835 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a8709cbe:::obj458:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162000 con 0x3752000 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.358+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a8709cbe:::obj458:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.358+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a8709cbe:::obj458:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.358+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.358+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.358+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a8709cbe:::obj458:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.358+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.358+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.358+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.358+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a87532ae:::obj194:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a87532ae:::obj194:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.358+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a87532ae:::obj194:head 2026-02-11T23:31:17.358+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a87532ae:::obj194:head v 107'194 2026-02-11T23:31:17.358+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.358+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.358+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.358+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.358+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a8709cbe:::obj458:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a8709cbe:::obj458:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 112/112 lb 1:a8709cbe:::obj458:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a8709cbe:::obj458:head) v3 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a6f174aa:::obj203:head local-lis/les=109/110 n=397 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a8709cbe:::obj458:head) 2026-02-11T23:31:17.358+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a8709cbe:::obj458:head) v3 finish latency 0.000015 2026-02-11T23:31:17.358+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.358+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.358+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.358+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6e600 con 0x3752000 2026-02-11T23:31:17.358+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a87532ae:::obj194:head)]) -- 0x4f51c80 con 0x3752000 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.358+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 836 ==== MOSDPGPush(1.0 112/109 [PushOp(1:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182b60 con 0x3752000 2026-02-11T23:31:17.358+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.358+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.358+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.358+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.358+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 837 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a87532ae:::obj194:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162580 con 0x3752000 2026-02-11T23:31:17.358+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.358+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a87532ae:::obj194:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.358+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.358+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a87532ae:::obj194:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.358+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a87532ae:::obj194:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:a8d6b24d:::obj229:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.358+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:a8d6b24d:::obj229:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:a8d6b24d:::obj229:head 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:a8d6b24d:::obj229:head v 107'229 2026-02-11T23:31:17.358+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.359+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.359+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.359+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a87532ae:::obj194:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a87532ae:::obj194:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 112/112 lb 1:a87532ae:::obj194:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a87532ae:::obj194:head) v3 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8709cbe:::obj458:head local-lis/les=109/110 n=398 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a87532ae:::obj194:head) 2026-02-11T23:31:17.359+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a87532ae:::obj194:head) v3 finish latency 0.000015 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.359+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.359+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6e180 con 0x3752000 2026-02-11T23:31:17.359+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:a8d6b24d:::obj229:head)]) -- 0x4f6fc80 con 0x3752000 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.359+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 838 ==== MOSDPGPush(1.0 112/109 [PushOp(1:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cd00 con 0x3752000 2026-02-11T23:31:17.359+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.359+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.359+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.359+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.359+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 839 ==== pg_backfill(progress 1.0 e 112/112 lb 1:a8d6b24d:::obj229:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3752000 2026-02-11T23:31:17.359+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:a8d6b24d:::obj229:head) v3 prio 15 type 95 cost 0 latency 0.000014 epoch 112 2026-02-11T23:31:17.359+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.359+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a8d6b24d:::obj229:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.359+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.359+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 112/109 [PushOp(1:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.359+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.359+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a8d6b24d:::obj229:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.359+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.359+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:aa1e2680:::obj587:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.359+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:aa1e2680:::obj587:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.359+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:aa1e2680:::obj587:head 2026-02-11T23:31:17.359+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:aa1e2680:::obj587:head v 107'587 2026-02-11T23:31:17.359+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.359+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.359+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.359+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:a8d6b24d:::obj229:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a8d6b24d:::obj229:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 112/112 lb 1:a8d6b24d:::obj229:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:a8d6b24d:::obj229:head) v3 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a87532ae:::obj194:head local-lis/les=109/110 n=399 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:a8d6b24d:::obj229:head) 2026-02-11T23:31:17.359+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:a8d6b24d:::obj229:head) v3 finish latency 0.000012 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.359+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.359+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.359+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f53800 con 0x3752000 2026-02-11T23:31:17.359+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:aa1e2680:::obj587:head)]) -- 0x4f6fb00 con 0x3752000 2026-02-11T23:31:17.359+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.360+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 840 ==== MOSDPGPush(1.0 112/109 [PushOp(1:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3752000 2026-02-11T23:31:17.360+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.360+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.360+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 841 ==== pg_backfill(progress 1.0 e 112/112 lb 1:aa1e2680:::obj587:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163080 con 0x3752000 2026-02-11T23:31:17.360+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:aa1e2680:::obj587:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.360+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:aa1e2680:::obj587:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:aa1e2680:::obj587:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ab0f5189:::obj85:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ab0f5189:::obj85:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ab0f5189:::obj85:head 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ab0f5189:::obj85:head v 107'85 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.360+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:aa1e2680:::obj587:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:aa1e2680:::obj587:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 112/112 lb 1:aa1e2680:::obj587:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:aa1e2680:::obj587:head) v3 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:a8d6b24d:::obj229:head local-lis/les=109/110 n=400 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:aa1e2680:::obj587:head) 2026-02-11T23:31:17.360+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:aa1e2680:::obj587:head) v3 finish latency 0.000010 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.360+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6e300 con 0x3752000 2026-02-11T23:31:17.360+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:ab0f5189:::obj85:head)]) -- 0x4f53e00 con 0x3752000 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.360+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 842 ==== MOSDPGPush(1.0 112/109 [PushOp(1:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516d6c0 con 0x3752000 2026-02-11T23:31:17.360+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.360+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.360+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 843 ==== pg_backfill(progress 1.0 e 112/112 lb 1:ab0f5189:::obj85:head) ==== 1061+0+0 (unknown 0 0 0) 0x5163600 con 0x3752000 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.360+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:ab0f5189:::obj85:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.360+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ab0f5189:::obj85:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ab0f5189:::obj85:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ab946124:::obj15:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ab946124:::obj15:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ab946124:::obj15:head 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ab946124:::obj15:head v 107'15 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.360+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.360+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ab0f5189:::obj85:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ab0f5189:::obj85:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 112/112 lb 1:ab0f5189:::obj85:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:ab0f5189:::obj85:head) v3 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aa1e2680:::obj587:head local-lis/les=109/110 n=401 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:ab0f5189:::obj85:head) 2026-02-11T23:31:17.360+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ab0f5189:::obj85:head) v3 finish latency 0.000011 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.360+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.360+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.360+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6f080 con 0x3752000 2026-02-11T23:31:17.360+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:ab946124:::obj15:head)]) -- 0x4fa1080 con 0x3752000 2026-02-11T23:31:17.360+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.361+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 844 ==== MOSDPGPush(1.0 112/109 [PushOp(1:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3752000 2026-02-11T23:31:17.361+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.361+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.361+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.361+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 845 ==== pg_backfill(progress 1.0 e 112/112 lb 1:ab946124:::obj15:head) ==== 1061+0+0 (unknown 0 0 0) 0x5163b80 con 0x3752000 2026-02-11T23:31:17.361+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:ab946124:::obj15:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.361+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.361+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ab946124:::obj15:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.361+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.361+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 112/109 [PushOp(1:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ab946124:::obj15:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ac7304c6:::obj483:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.361+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ac7304c6:::obj483:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ac7304c6:::obj483:head 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ac7304c6:::obj483:head v 107'483 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.361+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.361+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ab946124:::obj15:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.361+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ab946124:::obj15:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 112/112 lb 1:ab946124:::obj15:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.361+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.361+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:ab946124:::obj15:head) v3 2026-02-11T23:31:17.361+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab0f5189:::obj85:head local-lis/les=109/110 n=402 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:ab946124:::obj15:head) 2026-02-11T23:31:17.361+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.361+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ab946124:::obj15:head) v3 finish latency 0.000011 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.361+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.361+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.361+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6ec00 con 0x3752000 2026-02-11T23:31:17.361+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:ac7304c6:::obj483:head)]) -- 0x4f88180 con 0x3752000 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.361+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 846 ==== MOSDPGPush(1.0 112/109 [PushOp(1:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51821a0 con 0x3752000 2026-02-11T23:31:17.361+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.361+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.361+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.361+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.361+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 847 ==== pg_backfill(progress 1.0 e 112/112 lb 1:ac7304c6:::obj483:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3752000 2026-02-11T23:31:17.361+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.361+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:ac7304c6:::obj483:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 112 2026-02-11T23:31:17.361+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ac7304c6:::obj483:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.361+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 112/109 [PushOp(1:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.361+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ac7304c6:::obj483:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.361+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:acb8a831:::obj238:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.362+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:acb8a831:::obj238:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:acb8a831:::obj238:head 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:acb8a831:::obj238:head v 107'238 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.362+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.362+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ac7304c6:::obj483:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ac7304c6:::obj483:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 1.0 e 112/112 lb 1:ac7304c6:::obj483:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:ac7304c6:::obj483:head) v3 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ab946124:::obj15:head local-lis/les=109/110 n=403 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:ac7304c6:::obj483:head) 2026-02-11T23:31:17.362+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ac7304c6:::obj483:head) v3 finish latency 0.000013 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.362+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.362+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5800 con 0x3752000 2026-02-11T23:31:17.362+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:acb8a831:::obj238:head)]) -- 0x4f53e00 con 0x3752000 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.362+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 848 ==== MOSDPGPush(1.0 112/109 [PushOp(1:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5182680 con 0x3752000 2026-02-11T23:31:17.362+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.362+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.362+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.362+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 849 ==== pg_backfill(progress 1.0 e 112/112 lb 1:acb8a831:::obj238:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3752000 2026-02-11T23:31:17.362+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.362+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:acb8a831:::obj238:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.362+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:acb8a831:::obj238:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.362+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.362+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 112/109 [PushOp(1:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:acb8a831:::obj238:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:acfc8dd9:::obj74:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.362+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:acfc8dd9:::obj74:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:acfc8dd9:::obj74:head 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:acfc8dd9:::obj74:head v 107'74 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.362+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.362+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.362+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:acb8a831:::obj238:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:acb8a831:::obj238:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 112/112 lb 1:acb8a831:::obj238:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:acb8a831:::obj238:head) v3 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ac7304c6:::obj483:head local-lis/les=109/110 n=404 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:acb8a831:::obj238:head) 2026-02-11T23:31:17.362+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:acb8a831:::obj238:head) v3 finish latency 0.000012 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.362+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.362+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.362+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6ed80 con 0x3752000 2026-02-11T23:31:17.362+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:acfc8dd9:::obj74:head)]) -- 0x4f6fb00 con 0x3752000 2026-02-11T23:31:17.362+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.363+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 850 ==== MOSDPGPush(1.0 112/109 [PushOp(1:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182b60 con 0x3752000 2026-02-11T23:31:17.363+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.363+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.363+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 851 ==== pg_backfill(progress 1.0 e 112/112 lb 1:acfc8dd9:::obj74:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3752000 2026-02-11T23:31:17.363+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:acfc8dd9:::obj74:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 112 2026-02-11T23:31:17.363+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:acfc8dd9:::obj74:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 112/109 [PushOp(1:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:acfc8dd9:::obj74:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ad830abd:::obj496:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ad830abd:::obj496:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ad830abd:::obj496:head 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ad830abd:::obj496:head v 107'496 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.363+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:acfc8dd9:::obj74:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.363+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:acfc8dd9:::obj74:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 112/112 lb 1:acfc8dd9:::obj74:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.363+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.363+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:acfc8dd9:::obj74:head) v3 2026-02-11T23:31:17.363+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acb8a831:::obj238:head local-lis/les=109/110 n=405 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:acfc8dd9:::obj74:head) 2026-02-11T23:31:17.363+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.363+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:acfc8dd9:::obj74:head) v3 finish latency 0.000010 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.363+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.363+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6f200 con 0x3752000 2026-02-11T23:31:17.363+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:ad830abd:::obj496:head)]) -- 0x4f53800 con 0x3752000 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.363+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 852 ==== MOSDPGPush(1.0 112/109 [PushOp(1:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183040 con 0x3752000 2026-02-11T23:31:17.363+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.363+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.363+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 853 ==== pg_backfill(progress 1.0 e 112/112 lb 1:ad830abd:::obj496:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3752000 2026-02-11T23:31:17.363+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:ad830abd:::obj496:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.363+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ad830abd:::obj496:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 112/109 [PushOp(1:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ad830abd:::obj496:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:adc1b1a4:::obj497:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:adc1b1a4:::obj497:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:adc1b1a4:::obj497:head 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:adc1b1a4:::obj497:head v 107'497 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.363+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.363+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.363+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ad830abd:::obj496:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.363+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ad830abd:::obj496:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 1.0 e 112/112 lb 1:ad830abd:::obj496:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.363+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.363+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:ad830abd:::obj496:head) v3 2026-02-11T23:31:17.363+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:acfc8dd9:::obj74:head local-lis/les=109/110 n=406 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:ad830abd:::obj496:head) 2026-02-11T23:31:17.363+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.363+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ad830abd:::obj496:head) v3 finish latency 0.000012 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.363+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.364+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.364+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6f500 con 0x3752000 2026-02-11T23:31:17.364+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:adc1b1a4:::obj497:head)]) -- 0x4f6e180 con 0x3752000 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.364+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 854 ==== MOSDPGPush(1.0 112/109 [PushOp(1:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183520 con 0x3752000 2026-02-11T23:31:17.364+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.364+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.364+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 855 ==== pg_backfill(progress 1.0 e 112/112 lb 1:adc1b1a4:::obj497:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3752000 2026-02-11T23:31:17.364+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:adc1b1a4:::obj497:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 112 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.364+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:adc1b1a4:::obj497:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.364+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.364+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:adc1b1a4:::obj497:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.364+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.364+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.364+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ae2ef588:::obj234:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ae2ef588:::obj234:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.364+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ae2ef588:::obj234:head 2026-02-11T23:31:17.364+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ae2ef588:::obj234:head v 107'234 2026-02-11T23:31:17.364+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.364+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.364+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:adc1b1a4:::obj497:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.364+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:adc1b1a4:::obj497:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 112/112 lb 1:adc1b1a4:::obj497:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.364+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.364+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:adc1b1a4:::obj497:head) v3 2026-02-11T23:31:17.364+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ad830abd:::obj496:head local-lis/les=109/110 n=407 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:adc1b1a4:::obj497:head) 2026-02-11T23:31:17.364+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.364+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:adc1b1a4:::obj497:head) v3 finish latency 0.000011 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.364+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.364+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6f800 con 0x3752000 2026-02-11T23:31:17.364+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:ae2ef588:::obj234:head)]) -- 0x4f6e600 con 0x3752000 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.364+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 856 ==== MOSDPGPush(1.0 112/109 [PushOp(1:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3752000 2026-02-11T23:31:17.364+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.364+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.364+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 857 ==== pg_backfill(progress 1.0 e 112/112 lb 1:ae2ef588:::obj234:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d700 con 0x3752000 2026-02-11T23:31:17.364+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:ae2ef588:::obj234:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 112 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.364+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ae2ef588:::obj234:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.364+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ae2ef588:::obj234:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.364+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.364+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000048 MOSDPGPush(1.0 112/109 [PushOp(1:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:aec2b241:::obj347:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.365+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:aec2b241:::obj347:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:aec2b241:::obj347:head 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:aec2b241:::obj347:head v 107'347 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.365+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.365+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ae2ef588:::obj234:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ae2ef588:::obj234:head) v3 prio 15 cost 0 latency 0.000103 pg_backfill(progress 1.0 e 112/112 lb 1:ae2ef588:::obj234:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:ae2ef588:::obj234:head) v3 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:adc1b1a4:::obj497:head local-lis/les=109/110 n=408 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:ae2ef588:::obj234:head) 2026-02-11T23:31:17.365+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ae2ef588:::obj234:head) v3 finish latency 0.000013 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.365+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.365+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f6f680 con 0x3752000 2026-02-11T23:31:17.365+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:aec2b241:::obj347:head)]) -- 0x4f51b00 con 0x3752000 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.365+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 858 ==== MOSDPGPush(1.0 112/109 [PushOp(1:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3752000 2026-02-11T23:31:17.365+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.365+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.365+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.365+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 859 ==== pg_backfill(progress 1.0 e 112/112 lb 1:aec2b241:::obj347:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3752000 2026-02-11T23:31:17.365+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:aec2b241:::obj347:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 112 2026-02-11T23:31:17.365+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.365+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:aec2b241:::obj347:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.365+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.365+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:aec2b241:::obj347:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 112/109 [PushOp(1:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:aecaa835:::obj157:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.365+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:aecaa835:::obj157:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:aecaa835:::obj157:head 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:aecaa835:::obj157:head v 107'157 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.365+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.365+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.365+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:aec2b241:::obj347:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:aec2b241:::obj347:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 112/112 lb 1:aec2b241:::obj347:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:aec2b241:::obj347:head) v3 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:ae2ef588:::obj234:head local-lis/les=109/110 n=409 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:aec2b241:::obj347:head) 2026-02-11T23:31:17.365+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:aec2b241:::obj347:head) v3 finish latency 0.000011 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.365+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.365+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.365+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5e00 con 0x3752000 2026-02-11T23:31:17.365+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:aecaa835:::obj157:head)]) -- 0x4e71080 con 0x3752000 2026-02-11T23:31:17.365+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.366+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 860 ==== MOSDPGPush(1.0 112/109 [PushOp(1:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3752000 2026-02-11T23:31:17.366+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.366+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.366+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 861 ==== pg_backfill(progress 1.0 e 112/112 lb 1:aecaa835:::obj157:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3752000 2026-02-11T23:31:17.366+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:aecaa835:::obj157:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.366+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:aecaa835:::obj157:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 112/109 [PushOp(1:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:aecaa835:::obj157:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:af2b834d:::obj153:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:af2b834d:::obj153:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:af2b834d:::obj153:head 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:af2b834d:::obj153:head v 107'153 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.366+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:aecaa835:::obj157:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.366+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:aecaa835:::obj157:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 112/112 lb 1:aecaa835:::obj157:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.366+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.366+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:aecaa835:::obj157:head) v3 2026-02-11T23:31:17.366+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aec2b241:::obj347:head local-lis/les=109/110 n=410 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:aecaa835:::obj157:head) 2026-02-11T23:31:17.366+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.366+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:aecaa835:::obj157:head) v3 finish latency 0.000011 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.366+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.366+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e70180 con 0x3752000 2026-02-11T23:31:17.366+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:af2b834d:::obj153:head)]) -- 0x4f52f00 con 0x3752000 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.366+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 862 ==== MOSDPGPush(1.0 112/109 [PushOp(1:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3752000 2026-02-11T23:31:17.366+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.366+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.366+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 863 ==== pg_backfill(progress 1.0 e 112/112 lb 1:af2b834d:::obj153:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5b080 con 0x3752000 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.366+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:af2b834d:::obj153:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.366+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af2b834d:::obj153:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af2b834d:::obj153:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:af2dbe17:::obj414:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:af2dbe17:::obj414:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:af2dbe17:::obj414:head 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:af2dbe17:::obj414:head v 107'414 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.366+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.366+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.366+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af2b834d:::obj153:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.366+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af2b834d:::obj153:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 112/112 lb 1:af2b834d:::obj153:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.366+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.366+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:af2b834d:::obj153:head) v3 2026-02-11T23:31:17.366+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:aecaa835:::obj157:head local-lis/les=109/110 n=411 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:af2b834d:::obj153:head) 2026-02-11T23:31:17.366+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.366+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af2b834d:::obj153:head) v3 finish latency 0.000013 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.366+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.367+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.367+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e70d80 con 0x3752000 2026-02-11T23:31:17.367+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:af2dbe17:::obj414:head)]) -- 0x4f50a80 con 0x3752000 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.367+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 864 ==== MOSDPGPush(1.0 112/109 [PushOp(1:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3752000 2026-02-11T23:31:17.367+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.367+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.367+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 865 ==== pg_backfill(progress 1.0 e 112/112 lb 1:af2dbe17:::obj414:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3752000 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.367+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:af2dbe17:::obj414:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.367+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af2dbe17:::obj414:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af2dbe17:::obj414:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:af44a1d6:::obj35:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:af44a1d6:::obj35:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:af44a1d6:::obj35:head 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:af44a1d6:::obj35:head v 107'35 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.367+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af2dbe17:::obj414:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.367+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af2dbe17:::obj414:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 112/112 lb 1:af2dbe17:::obj414:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.367+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.367+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:af2dbe17:::obj414:head) v3 2026-02-11T23:31:17.367+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2b834d:::obj153:head local-lis/les=109/110 n=412 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:af2dbe17:::obj414:head) 2026-02-11T23:31:17.367+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.367+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af2dbe17:::obj414:head) v3 finish latency 0.000012 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.367+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.367+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96180 con 0x3752000 2026-02-11T23:31:17.367+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:af44a1d6:::obj35:head)]) -- 0x4f50f00 con 0x3752000 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.367+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 866 ==== MOSDPGPush(1.0 112/109 [PushOp(1:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43f2000 con 0x3752000 2026-02-11T23:31:17.367+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.367+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.367+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 867 ==== pg_backfill(progress 1.0 e 112/112 lb 1:af44a1d6:::obj35:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5b600 con 0x3752000 2026-02-11T23:31:17.367+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:af44a1d6:::obj35:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 112 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.367+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af44a1d6:::obj35:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.367+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.367+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af44a1d6:::obj35:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 112/109 [PushOp(1:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.367+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:af5f95cb:::obj12:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.368+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:af5f95cb:::obj12:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:af5f95cb:::obj12:head 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:af5f95cb:::obj12:head v 107'12 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.368+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:17.368+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af44a1d6:::obj35:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af44a1d6:::obj35:head) v3 prio 15 cost 0 latency 0.000101 pg_backfill(progress 1.0 e 112/112 lb 1:af44a1d6:::obj35:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:af44a1d6:::obj35:head) v3 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af2dbe17:::obj414:head local-lis/les=109/110 n=413 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:af44a1d6:::obj35:head) 2026-02-11T23:31:17.368+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af44a1d6:::obj35:head) v3 finish latency 0.000018 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.368+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.368+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe5b00 con 0x3752000 2026-02-11T23:31:17.368+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:af5f95cb:::obj12:head)]) -- 0x4e71500 con 0x3752000 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.368+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 868 ==== MOSDPGPush(1.0 112/109 [PushOp(1:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3752000 2026-02-11T23:31:17.368+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.368+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.368+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.368+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 869 ==== pg_backfill(progress 1.0 e 112/112 lb 1:af5f95cb:::obj12:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5bb80 con 0x3752000 2026-02-11T23:31:17.368+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.368+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:af5f95cb:::obj12:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.368+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.368+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af5f95cb:::obj12:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.368+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af5f95cb:::obj12:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:af93a04b:::obj437:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.368+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:af93a04b:::obj437:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:af93a04b:::obj437:head 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:af93a04b:::obj437:head v 107'437 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.368+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.368+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.368+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af5f95cb:::obj12:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af5f95cb:::obj12:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 112/112 lb 1:af5f95cb:::obj12:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:af5f95cb:::obj12:head) v3 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af44a1d6:::obj35:head local-lis/les=109/110 n=414 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:af5f95cb:::obj12:head) 2026-02-11T23:31:17.368+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af5f95cb:::obj12:head) v3 finish latency 0.000012 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.368+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.368+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.368+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e70480 con 0x3752000 2026-02-11T23:31:17.368+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.368+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:af93a04b:::obj437:head)]) -- 0x4f52480 con 0x3752000 2026-02-11T23:31:17.368+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.369+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 870 ==== MOSDPGPush(1.0 112/109 [PushOp(1:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3752000 2026-02-11T23:31:17.369+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.369+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.369+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 871 ==== pg_backfill(progress 1.0 e 112/112 lb 1:af93a04b:::obj437:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3752000 2026-02-11T23:31:17.369+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:af93a04b:::obj437:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.369+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af93a04b:::obj437:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af93a04b:::obj437:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:af9ff462:::obj165:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:af9ff462:::obj165:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:af9ff462:::obj165:head 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:af9ff462:::obj165:head v 107'165 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.369+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af93a04b:::obj437:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.369+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af93a04b:::obj437:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 112/112 lb 1:af93a04b:::obj437:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.369+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.369+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:af93a04b:::obj437:head) v3 2026-02-11T23:31:17.369+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af5f95cb:::obj12:head local-lis/les=109/110 n=415 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:af93a04b:::obj437:head) 2026-02-11T23:31:17.369+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.369+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af93a04b:::obj437:head) v3 finish latency 0.000013 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.369+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.369+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa0d80 con 0x3752000 2026-02-11T23:31:17.369+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:af9ff462:::obj165:head)]) -- 0x4f51380 con 0x3752000 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.369+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 872 ==== MOSDPGPush(1.0 112/109 [PushOp(1:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3752000 2026-02-11T23:31:17.369+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.369+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.369+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 873 ==== pg_backfill(progress 1.0 e 112/112 lb 1:af9ff462:::obj165:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3752000 2026-02-11T23:31:17.369+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:af9ff462:::obj165:head) v3 prio 15 type 95 cost 0 latency 0.000017 epoch 112 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.369+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af9ff462:::obj165:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af9ff462:::obj165:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:afacbcad:::obj315:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:afacbcad:::obj315:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:afacbcad:::obj315:head 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:afacbcad:::obj315:head v 107'315 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.369+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.369+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.369+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:af9ff462:::obj165:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.369+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af9ff462:::obj165:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 112/112 lb 1:af9ff462:::obj165:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.369+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.369+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:af9ff462:::obj165:head) v3 2026-02-11T23:31:17.369+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af93a04b:::obj437:head local-lis/les=109/110 n=416 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:af9ff462:::obj165:head) 2026-02-11T23:31:17.369+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.369+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:af9ff462:::obj165:head) v3 finish latency 0.000012 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.369+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.370+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.370+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.370+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96d80 con 0x3752000 2026-02-11T23:31:17.370+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:afacbcad:::obj315:head)]) -- 0x4f52780 con 0x3752000 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.370+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 874 ==== MOSDPGPush(1.0 112/109 [PushOp(1:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51836c0 con 0x3752000 2026-02-11T23:31:17.370+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.370+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.370+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.370+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.370+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 875 ==== pg_backfill(progress 1.0 e 112/112 lb 1:afacbcad:::obj315:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbec00 con 0x3752000 2026-02-11T23:31:17.370+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.370+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:afacbcad:::obj315:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.370+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.370+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:afacbcad:::obj315:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:afacbcad:::obj315:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.370+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 112/109 [PushOp(1:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.370+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.370+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.370+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.370+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:afbdc8ea:::obj55:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.370+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:afbdc8ea:::obj55:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.370+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:afbdc8ea:::obj55:head 2026-02-11T23:31:17.370+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:afbdc8ea:::obj55:head v 107'55 2026-02-11T23:31:17.370+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.370+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.370+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.370+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:afacbcad:::obj315:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.370+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:afacbcad:::obj315:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 112/112 lb 1:afacbcad:::obj315:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.370+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.370+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:afacbcad:::obj315:head) v3 2026-02-11T23:31:17.370+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:af9ff462:::obj165:head local-lis/les=109/110 n=417 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:afacbcad:::obj315:head) 2026-02-11T23:31:17.370+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.370+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:afacbcad:::obj315:head) v3 finish latency 0.000013 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.370+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.370+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.370+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa0c00 con 0x3752000 2026-02-11T23:31:17.370+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:afbdc8ea:::obj55:head)]) -- 0x4f51680 con 0x3752000 2026-02-11T23:31:17.370+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.371+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 876 ==== MOSDPGPush(1.0 112/109 [PushOp(1:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51831e0 con 0x3752000 2026-02-11T23:31:17.371+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.371+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.371+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 877 ==== pg_backfill(progress 1.0 e 112/112 lb 1:afbdc8ea:::obj55:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3752000 2026-02-11T23:31:17.371+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:afbdc8ea:::obj55:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.371+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:afbdc8ea:::obj55:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:afbdc8ea:::obj55:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:afd348d8:::obj184:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:afd348d8:::obj184:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:afd348d8:::obj184:head 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:afd348d8:::obj184:head v 107'184 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.371+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:afbdc8ea:::obj55:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:afbdc8ea:::obj55:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 112/112 lb 1:afbdc8ea:::obj55:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:afbdc8ea:::obj55:head) v3 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afacbcad:::obj315:head local-lis/les=109/110 n=418 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:afbdc8ea:::obj55:head) 2026-02-11T23:31:17.371+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:afbdc8ea:::obj55:head) v3 finish latency 0.000012 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.371+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fe4c00 con 0x3752000 2026-02-11T23:31:17.371+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:afd348d8:::obj184:head)]) -- 0x4e71200 con 0x3752000 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.371+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 878 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182d00 con 0x3752000 2026-02-11T23:31:17.371+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.371+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.371+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 879 ==== pg_backfill(progress 1.0 e 112/112 lb 1:afd348d8:::obj184:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3752000 2026-02-11T23:31:17.371+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:afd348d8:::obj184:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.371+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:afd348d8:::obj184:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:afd348d8:::obj184:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b0395c4b:::obj158:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b0395c4b:::obj158:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b0395c4b:::obj158:head 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b0395c4b:::obj158:head v 107'158 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.371+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.371+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:afd348d8:::obj184:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:afd348d8:::obj184:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 112/112 lb 1:afd348d8:::obj184:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:afd348d8:::obj184:head) v3 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afbdc8ea:::obj55:head local-lis/les=109/110 n=419 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:afd348d8:::obj184:head) 2026-02-11T23:31:17.371+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:afd348d8:::obj184:head) v3 finish latency 0.000011 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.371+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.371+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.371+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4fa0a80 con 0x3752000 2026-02-11T23:31:17.371+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b0395c4b:::obj158:head)]) -- 0x4f51980 con 0x3752000 2026-02-11T23:31:17.371+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.372+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 880 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182820 con 0x3752000 2026-02-11T23:31:17.372+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.372+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.372+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 881 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b0395c4b:::obj158:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3752000 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.372+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b0395c4b:::obj158:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.372+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b0395c4b:::obj158:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b0395c4b:::obj158:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b133d16d:::obj222:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b133d16d:::obj222:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b133d16d:::obj222:head 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b133d16d:::obj222:head v 107'222 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.372+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b0395c4b:::obj158:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.372+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b0395c4b:::obj158:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 112/112 lb 1:b0395c4b:::obj158:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.372+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.372+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b0395c4b:::obj158:head) v3 2026-02-11T23:31:17.372+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:afd348d8:::obj184:head local-lis/les=109/110 n=420 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b0395c4b:::obj158:head) 2026-02-11T23:31:17.372+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.372+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b0395c4b:::obj158:head) v3 finish latency 0.000011 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.372+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.372+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4d96a80 con 0x3752000 2026-02-11T23:31:17.372+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b133d16d:::obj222:head)]) -- 0x4fe5c80 con 0x3752000 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.372+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 882 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3752000 2026-02-11T23:31:17.372+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.372+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.372+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 883 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b133d16d:::obj222:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3752000 2026-02-11T23:31:17.372+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b133d16d:::obj222:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.372+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b133d16d:::obj222:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b133d16d:::obj222:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b164ceff:::obj421:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b164ceff:::obj421:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b164ceff:::obj421:head 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b164ceff:::obj421:head v 107'421 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.372+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.372+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.372+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b133d16d:::obj222:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.372+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b133d16d:::obj222:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 112/112 lb 1:b133d16d:::obj222:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.372+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.372+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b133d16d:::obj222:head) v3 2026-02-11T23:31:17.372+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b0395c4b:::obj158:head local-lis/les=109/110 n=421 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b133d16d:::obj222:head) 2026-02-11T23:31:17.372+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.372+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b133d16d:::obj222:head) v3 finish latency 0.000011 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.372+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.373+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.373+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.373+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4e70a80 con 0x3752000 2026-02-11T23:31:17.373+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b164ceff:::obj421:head)]) -- 0x4fa0c00 con 0x3752000 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.373+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 884 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dd40 con 0x3752000 2026-02-11T23:31:17.373+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.373+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.373+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 885 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b164ceff:::obj421:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3752000 2026-02-11T23:31:17.373+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b164ceff:::obj421:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.373+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b164ceff:::obj421:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.373+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(1.0 112/109 [PushOp(1:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.373+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.373+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.373+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.373+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b164ceff:::obj421:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.373+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.373+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b1721c5a:::obj526:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b1721c5a:::obj526:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.373+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b1721c5a:::obj526:head 2026-02-11T23:31:17.373+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b1721c5a:::obj526:head v 107'526 2026-02-11T23:31:17.373+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.373+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.373+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.373+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.373+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b164ceff:::obj421:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.373+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b164ceff:::obj421:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 1.0 e 112/112 lb 1:b164ceff:::obj421:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.373+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.373+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b164ceff:::obj421:head) v3 2026-02-11T23:31:17.373+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b133d16d:::obj222:head local-lis/les=109/110 n=422 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b164ceff:::obj421:head) 2026-02-11T23:31:17.373+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.373+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b164ceff:::obj421:head) v3 finish latency 0.000013 2026-02-11T23:31:17.373+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.373+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.373+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.373+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f50d80 con 0x3752000 2026-02-11T23:31:17.373+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b1721c5a:::obj526:head)]) -- 0x4e70780 con 0x3752000 2026-02-11T23:31:17.373+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.374+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 886 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d860 con 0x3752000 2026-02-11T23:31:17.374+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.374+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.374+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 887 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b1721c5a:::obj526:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3752000 2026-02-11T23:31:17.374+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b1721c5a:::obj526:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.374+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b1721c5a:::obj526:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b1721c5a:::obj526:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b1ca68cf:::obj358:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b1ca68cf:::obj358:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b1ca68cf:::obj358:head 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b1ca68cf:::obj358:head v 107'358 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.374+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b1721c5a:::obj526:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b1721c5a:::obj526:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 112/112 lb 1:b1721c5a:::obj526:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b1721c5a:::obj526:head) v3 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b164ceff:::obj421:head local-lis/les=109/110 n=423 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b1721c5a:::obj526:head) 2026-02-11T23:31:17.374+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b1721c5a:::obj526:head) v3 finish latency 0.000015 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.374+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f52180 con 0x3752000 2026-02-11T23:31:17.374+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b1ca68cf:::obj358:head)]) -- 0x4d96d80 con 0x3752000 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.374+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 888 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d380 con 0x3752000 2026-02-11T23:31:17.374+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.374+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.374+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 889 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b1ca68cf:::obj358:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3752000 2026-02-11T23:31:17.374+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b1ca68cf:::obj358:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.374+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b1ca68cf:::obj358:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b1ca68cf:::obj358:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b1e2ea00:::obj289:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b1e2ea00:::obj289:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b1e2ea00:::obj289:head 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b1e2ea00:::obj289:head v 107'289 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.374+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.374+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b1ca68cf:::obj358:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b1ca68cf:::obj358:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 112/112 lb 1:b1ca68cf:::obj358:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b1ca68cf:::obj358:head) v3 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1721c5a:::obj526:head local-lis/les=109/110 n=424 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b1ca68cf:::obj358:head) 2026-02-11T23:31:17.374+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b1ca68cf:::obj358:head) v3 finish latency 0.000012 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.374+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.374+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.374+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f50900 con 0x3752000 2026-02-11T23:31:17.374+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b1e2ea00:::obj289:head)]) -- 0x4fa0d80 con 0x3752000 2026-02-11T23:31:17.374+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.375+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 890 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cea0 con 0x3752000 2026-02-11T23:31:17.375+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.375+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.375+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 891 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b1e2ea00:::obj289:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3752000 2026-02-11T23:31:17.375+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b1e2ea00:::obj289:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.375+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b1e2ea00:::obj289:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b1e2ea00:::obj289:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b26ae718:::obj317:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b26ae718:::obj317:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b26ae718:::obj317:head 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b26ae718:::obj317:head v 107'317 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.375+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b1e2ea00:::obj289:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.375+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b1e2ea00:::obj289:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 112/112 lb 1:b1e2ea00:::obj289:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.375+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.375+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b1e2ea00:::obj289:head) v3 2026-02-11T23:31:17.375+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1ca68cf:::obj358:head local-lis/les=109/110 n=425 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b1e2ea00:::obj289:head) 2026-02-11T23:31:17.375+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.375+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b1e2ea00:::obj289:head) v3 finish latency 0.000011 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.375+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.375+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f50480 con 0x3752000 2026-02-11T23:31:17.375+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b26ae718:::obj317:head)]) -- 0x4e70480 con 0x3752000 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.375+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 892 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51829c0 con 0x3752000 2026-02-11T23:31:17.375+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.375+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.375+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 893 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b26ae718:::obj317:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3752000 2026-02-11T23:31:17.375+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b26ae718:::obj317:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 112 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.375+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b26ae718:::obj317:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(1.0 112/109 [PushOp(1:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.375+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b26ae718:::obj317:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b2b26856:::obj586:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.375+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b2b26856:::obj586:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b2b26856:::obj586:head 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b2b26856:::obj586:head v 107'586 2026-02-11T23:31:17.375+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.375+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.376+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.376+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b26ae718:::obj317:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b26ae718:::obj317:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 1.0 e 112/112 lb 1:b26ae718:::obj317:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b26ae718:::obj317:head) v3 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b1e2ea00:::obj289:head local-lis/les=109/110 n=426 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b26ae718:::obj317:head) 2026-02-11T23:31:17.376+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b26ae718:::obj317:head) v3 finish latency 0.000012 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.376+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.376+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f50780 con 0x3752000 2026-02-11T23:31:17.376+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b2b26856:::obj586:head)]) -- 0x4fe5b00 con 0x3752000 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.376+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 894 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182ea0 con 0x3752000 2026-02-11T23:31:17.376+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.376+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.376+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.376+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 895 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b2b26856:::obj586:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3752000 2026-02-11T23:31:17.376+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b2b26856:::obj586:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 112 2026-02-11T23:31:17.376+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.376+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b2b26856:::obj586:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.376+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.376+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.376+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.376+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b2b26856:::obj586:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.376+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.376+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.376+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b2ec4718:::obj500:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.376+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b2ec4718:::obj500:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.376+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b2ec4718:::obj500:head 2026-02-11T23:31:17.376+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b2ec4718:::obj500:head v 107'500 2026-02-11T23:31:17.376+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.376+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.376+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.376+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b2b26856:::obj586:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b2b26856:::obj586:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 112/112 lb 1:b2b26856:::obj586:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b2b26856:::obj586:head) v3 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b26ae718:::obj317:head local-lis/les=109/110 n=427 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b2b26856:::obj586:head) 2026-02-11T23:31:17.376+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b2b26856:::obj586:head) v3 finish latency 0.000014 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.376+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.376+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.376+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x49b9e00 con 0x3752000 2026-02-11T23:31:17.376+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b2ec4718:::obj500:head)]) -- 0x4d96180 con 0x3752000 2026-02-11T23:31:17.376+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.377+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 896 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183380 con 0x3752000 2026-02-11T23:31:17.377+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.377+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.377+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 897 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b2ec4718:::obj500:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded080 con 0x3752000 2026-02-11T23:31:17.377+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b2ec4718:::obj500:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.377+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b2ec4718:::obj500:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 112/109 [PushOp(1:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b2ec4718:::obj500:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b33086a8:::obj36:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b33086a8:::obj36:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b33086a8:::obj36:head 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b33086a8:::obj36:head v 107'36 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.377+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b2ec4718:::obj500:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.377+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b2ec4718:::obj500:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 112/112 lb 1:b2ec4718:::obj500:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.377+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.377+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b2ec4718:::obj500:head) v3 2026-02-11T23:31:17.377+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2b26856:::obj586:head local-lis/les=109/110 n=428 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b2ec4718:::obj500:head) 2026-02-11T23:31:17.377+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.377+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b2ec4718:::obj500:head) v3 finish latency 0.000014 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.377+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2de00 con 0x3752000 2026-02-11T23:31:17.377+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b33086a8:::obj36:head)]) -- 0x4f50300 con 0x3752000 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.377+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 898 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3752000 2026-02-11T23:31:17.377+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.377+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.377+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 899 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b33086a8:::obj36:head) ==== 1061+0+0 (unknown 0 0 0) 0x4ded600 con 0x3752000 2026-02-11T23:31:17.377+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b33086a8:::obj36:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 112 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.377+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b33086a8:::obj36:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b33086a8:::obj36:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b4726009:::obj503:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b4726009:::obj503:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b4726009:::obj503:head 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b4726009:::obj503:head v 107'503 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.377+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b33086a8:::obj36:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.377+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b33086a8:::obj36:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 112/112 lb 1:b33086a8:::obj36:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.377+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.377+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b33086a8:::obj36:head) v3 2026-02-11T23:31:17.377+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b2ec4718:::obj500:head local-lis/les=109/110 n=429 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b33086a8:::obj36:head) 2026-02-11T23:31:17.377+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.377+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b33086a8:::obj36:head) v3 finish latency 0.000010 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.377+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.377+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.377+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2dc80 con 0x3752000 2026-02-11T23:31:17.377+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b4726009:::obj503:head)]) -- 0x4f50180 con 0x3752000 2026-02-11T23:31:17.377+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.378+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 900 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183d40 con 0x3752000 2026-02-11T23:31:17.378+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.378+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.378+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.378+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 901 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b4726009:::obj503:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3752000 2026-02-11T23:31:17.378+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b4726009:::obj503:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.378+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b4726009:::obj503:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.378+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.378+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.378+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 112/109 [PushOp(1:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b4726009:::obj503:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b5298834:::obj386:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.378+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b5298834:::obj386:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b5298834:::obj386:head 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b5298834:::obj386:head v 107'386 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.378+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.378+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b4726009:::obj503:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.378+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b4726009:::obj503:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 112/112 lb 1:b4726009:::obj503:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.378+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.378+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b4726009:::obj503:head) v3 2026-02-11T23:31:17.378+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b33086a8:::obj36:head local-lis/les=109/110 n=430 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b4726009:::obj503:head) 2026-02-11T23:31:17.378+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.378+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b4726009:::obj503:head) v3 finish latency 0.000014 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.378+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.378+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.378+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2d500 con 0x3752000 2026-02-11T23:31:17.378+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b5298834:::obj386:head)]) -- 0x4e70d80 con 0x3752000 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.378+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 902 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3ba0 con 0x3752000 2026-02-11T23:31:17.378+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.378+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.378+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 903 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b5298834:::obj386:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3752000 2026-02-11T23:31:17.378+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.378+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5298834:::obj386:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.378+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5298834:::obj386:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.378+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.378+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.378+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.378+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5298834:::obj386:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.378+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b5339e65:::obj355:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.379+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b5339e65:::obj355:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b5339e65:::obj355:head 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b5339e65:::obj355:head v 107'355 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.379+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.379+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5298834:::obj386:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5298834:::obj386:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 112/112 lb 1:b5298834:::obj386:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b5298834:::obj386:head) v3 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b4726009:::obj503:head local-lis/les=109/110 n=431 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b5298834:::obj386:head) 2026-02-11T23:31:17.379+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5298834:::obj386:head) v3 finish latency 0.000012 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.379+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.379+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2d800 con 0x3752000 2026-02-11T23:31:17.379+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b5339e65:::obj355:head)]) -- 0x4e70180 con 0x3752000 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.379+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 904 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3752000 2026-02-11T23:31:17.379+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.379+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.379+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.379+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.379+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.379+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.379+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 905 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b5339e65:::obj355:head) ==== 1062+0+0 (unknown 0 0 0) 0x4deec00 con 0x3752000 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.379+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5339e65:::obj355:head) v3 prio 15 type 95 cost 0 latency 0.000013 epoch 112 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.379+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5339e65:::obj355:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5339e65:::obj355:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b5445b0d:::obj455:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.379+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b5445b0d:::obj455:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b5445b0d:::obj455:head 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b5445b0d:::obj455:head v 107'455 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.379+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.379+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.379+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5339e65:::obj355:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5339e65:::obj355:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 112/112 lb 1:b5339e65:::obj355:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b5339e65:::obj355:head) v3 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5298834:::obj386:head local-lis/les=109/110 n=432 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b5339e65:::obj355:head) 2026-02-11T23:31:17.379+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5339e65:::obj355:head) v3 finish latency 0.000014 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.379+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.379+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.379+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x49b9c80 con 0x3752000 2026-02-11T23:31:17.379+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b5445b0d:::obj455:head)]) -- 0x4f51b00 con 0x3752000 2026-02-11T23:31:17.379+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.380+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 906 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3752000 2026-02-11T23:31:17.380+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.380+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.380+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 907 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b5445b0d:::obj455:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def180 con 0x3752000 2026-02-11T23:31:17.380+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5445b0d:::obj455:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.380+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5445b0d:::obj455:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 112/109 [PushOp(1:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5445b0d:::obj455:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b59cea89:::obj183:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b59cea89:::obj183:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b59cea89:::obj183:head 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b59cea89:::obj183:head v 107'183 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.380+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5445b0d:::obj455:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5445b0d:::obj455:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 112/112 lb 1:b5445b0d:::obj455:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b5445b0d:::obj455:head) v3 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5339e65:::obj355:head local-lis/les=109/110 n=433 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b5445b0d:::obj455:head) 2026-02-11T23:31:17.380+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5445b0d:::obj455:head) v3 finish latency 0.000012 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.380+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2d380 con 0x3752000 2026-02-11T23:31:17.380+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b59cea89:::obj183:head)]) -- 0x4f6e600 con 0x3752000 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.380+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 908 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3752000 2026-02-11T23:31:17.380+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.380+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.380+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 909 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b59cea89:::obj183:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164c00 con 0x3752000 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.380+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b59cea89:::obj183:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.380+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b59cea89:::obj183:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 112/109 [PushOp(1:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b59cea89:::obj183:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b5a67629:::obj219:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b5a67629:::obj219:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b5a67629:::obj219:head 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b5a67629:::obj219:head v 107'219 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.380+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.380+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b59cea89:::obj183:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b59cea89:::obj183:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 112/112 lb 1:b59cea89:::obj183:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b59cea89:::obj183:head) v3 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5445b0d:::obj455:head local-lis/les=109/110 n=434 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b59cea89:::obj183:head) 2026-02-11T23:31:17.380+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b59cea89:::obj183:head) v3 finish latency 0.000011 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.380+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.380+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.380+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2cf00 con 0x3752000 2026-02-11T23:31:17.380+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b5a67629:::obj219:head)]) -- 0x4f6e180 con 0x3752000 2026-02-11T23:31:17.380+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.381+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 910 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3752000 2026-02-11T23:31:17.381+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.381+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.381+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 911 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b5a67629:::obj219:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165180 con 0x3752000 2026-02-11T23:31:17.381+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5a67629:::obj219:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 112 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.381+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5a67629:::obj219:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5a67629:::obj219:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b609c5a3:::obj261:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b609c5a3:::obj261:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b609c5a3:::obj261:head 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b609c5a3:::obj261:head v 107'261 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.381+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b5a67629:::obj219:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.381+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5a67629:::obj219:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 112/112 lb 1:b5a67629:::obj219:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.381+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.381+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b5a67629:::obj219:head) v3 2026-02-11T23:31:17.381+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b59cea89:::obj183:head local-lis/les=109/110 n=435 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b5a67629:::obj219:head) 2026-02-11T23:31:17.381+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.381+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b5a67629:::obj219:head) v3 finish latency 0.000013 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.381+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.381+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2d200 con 0x3752000 2026-02-11T23:31:17.381+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b609c5a3:::obj261:head)]) -- 0x4f53800 con 0x3752000 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.381+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 912 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2340 con 0x3752000 2026-02-11T23:31:17.381+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.381+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.381+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 913 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b609c5a3:::obj261:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3752000 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.381+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b609c5a3:::obj261:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 112 2026-02-11T23:31:17.381+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b609c5a3:::obj261:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b609c5a3:::obj261:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b66eab34:::obj112:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b66eab34:::obj112:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b66eab34:::obj112:head 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b66eab34:::obj112:head v 107'112 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.381+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.381+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.381+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b609c5a3:::obj261:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.381+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b609c5a3:::obj261:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 112/112 lb 1:b609c5a3:::obj261:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.381+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.381+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b609c5a3:::obj261:head) v3 2026-02-11T23:31:17.381+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b5a67629:::obj219:head local-lis/les=109/110 n=436 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b609c5a3:::obj261:head) 2026-02-11T23:31:17.381+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.381+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b609c5a3:::obj261:head) v3 finish latency 0.000011 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.381+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.382+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.382+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.382+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2ca80 con 0x3752000 2026-02-11T23:31:17.382+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b66eab34:::obj112:head)]) -- 0x4f6fb00 con 0x3752000 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.382+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 914 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183d40 con 0x3752000 2026-02-11T23:31:17.382+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.382+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.382+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 915 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b66eab34:::obj112:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3752000 2026-02-11T23:31:17.382+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b66eab34:::obj112:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.382+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b66eab34:::obj112:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.382+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 112/109 [PushOp(1:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.382+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.382+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.382+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.382+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b66eab34:::obj112:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.382+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.382+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b6a90992:::obj552:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b6a90992:::obj552:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.382+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b6a90992:::obj552:head 2026-02-11T23:31:17.382+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b6a90992:::obj552:head v 107'552 2026-02-11T23:31:17.382+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.382+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.382+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.382+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.382+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b66eab34:::obj112:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.382+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b66eab34:::obj112:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 1.0 e 112/112 lb 1:b66eab34:::obj112:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.382+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.382+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b66eab34:::obj112:head) v3 2026-02-11T23:31:17.382+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b609c5a3:::obj261:head local-lis/les=109/110 n=437 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b66eab34:::obj112:head) 2026-02-11T23:31:17.382+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.382+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b66eab34:::obj112:head) v3 finish latency 0.000023 2026-02-11T23:31:17.382+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.382+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.382+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.382+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2c900 con 0x3752000 2026-02-11T23:31:17.382+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b6a90992:::obj552:head)]) -- 0x4f2cd80 con 0x3752000 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.382+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.382+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.383+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 916 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3752000 2026-02-11T23:31:17.383+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.383+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.383+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 917 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b6a90992:::obj552:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3752000 2026-02-11T23:31:17.383+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b6a90992:::obj552:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.383+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b6a90992:::obj552:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 112/109 [PushOp(1:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b6a90992:::obj552:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b6dfbaf7:::obj426:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b6dfbaf7:::obj426:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b6dfbaf7:::obj426:head 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b6dfbaf7:::obj426:head v 107'426 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.383+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b6a90992:::obj552:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b6a90992:::obj552:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 112/112 lb 1:b6a90992:::obj552:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b6a90992:::obj552:head) v3 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b66eab34:::obj112:head local-lis/les=109/110 n=438 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b6a90992:::obj552:head) 2026-02-11T23:31:17.383+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b6a90992:::obj552:head) v3 finish latency 0.000014 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.383+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2c180 con 0x3752000 2026-02-11T23:31:17.383+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b6dfbaf7:::obj426:head)]) -- 0x4f53e00 con 0x3752000 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.383+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 918 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183380 con 0x3752000 2026-02-11T23:31:17.383+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.383+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.383+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 919 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b6dfbaf7:::obj426:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5080 con 0x3752000 2026-02-11T23:31:17.383+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b6dfbaf7:::obj426:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.383+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b6dfbaf7:::obj426:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 112/109 [PushOp(1:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b6dfbaf7:::obj426:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b70eecc2:::obj59:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b70eecc2:::obj59:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b70eecc2:::obj59:head 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b70eecc2:::obj59:head v 107'59 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.383+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.383+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b6dfbaf7:::obj426:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b6dfbaf7:::obj426:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 112/112 lb 1:b6dfbaf7:::obj426:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b6dfbaf7:::obj426:head) v3 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6a90992:::obj552:head local-lis/les=109/110 n=439 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b6dfbaf7:::obj426:head) 2026-02-11T23:31:17.383+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b6dfbaf7:::obj426:head) v3 finish latency 0.000013 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.383+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.383+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.383+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2c480 con 0x3752000 2026-02-11T23:31:17.383+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b70eecc2:::obj59:head)]) -- 0x4f88180 con 0x3752000 2026-02-11T23:31:17.383+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.384+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 920 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182ea0 con 0x3752000 2026-02-11T23:31:17.384+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 112 2026-02-11T23:31:17.384+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.384+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 921 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b70eecc2:::obj59:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5600 con 0x3752000 2026-02-11T23:31:17.384+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b70eecc2:::obj59:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.384+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b70eecc2:::obj59:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b70eecc2:::obj59:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b75b9722:::obj495:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b75b9722:::obj495:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b75b9722:::obj495:head 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b75b9722:::obj495:head v 107'495 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.384+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b70eecc2:::obj59:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.384+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b70eecc2:::obj59:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 112/112 lb 1:b70eecc2:::obj59:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.384+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.384+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b70eecc2:::obj59:head) v3 2026-02-11T23:31:17.384+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b6dfbaf7:::obj426:head local-lis/les=109/110 n=440 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b70eecc2:::obj59:head) 2026-02-11T23:31:17.384+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.384+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b70eecc2:::obj59:head) v3 finish latency 0.000012 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.384+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.384+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4a05c80 con 0x3752000 2026-02-11T23:31:17.384+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b75b9722:::obj495:head)]) -- 0x4f6ec00 con 0x3752000 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.384+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 922 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51829c0 con 0x3752000 2026-02-11T23:31:17.384+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.384+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.384+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 923 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b75b9722:::obj495:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3752000 2026-02-11T23:31:17.384+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b75b9722:::obj495:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 112 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.384+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b75b9722:::obj495:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b75b9722:::obj495:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b841f58a:::obj361:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b841f58a:::obj361:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b841f58a:::obj361:head 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b841f58a:::obj361:head v 107'361 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.384+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.384+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.384+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b75b9722:::obj495:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.384+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b75b9722:::obj495:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 112/112 lb 1:b75b9722:::obj495:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.384+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.384+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b75b9722:::obj495:head) v3 2026-02-11T23:31:17.384+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b70eecc2:::obj59:head local-lis/les=109/110 n=441 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b75b9722:::obj495:head) 2026-02-11T23:31:17.384+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.384+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b75b9722:::obj495:head) v3 finish latency 0.000012 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.384+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.385+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.385+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.385+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4f2c000 con 0x3752000 2026-02-11T23:31:17.385+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b841f58a:::obj361:head)]) -- 0x4f6e600 con 0x3752000 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.385+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 924 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516cea0 con 0x3752000 2026-02-11T23:31:17.385+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.385+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.385+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 925 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b841f58a:::obj361:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3752000 2026-02-11T23:31:17.385+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b841f58a:::obj361:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.385+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.385+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b841f58a:::obj361:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.385+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.385+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.385+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.385+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 112/109 [PushOp(1:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.385+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.385+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b841f58a:::obj361:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.385+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.385+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b8510467:::obj40:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.385+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b8510467:::obj40:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.385+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b8510467:::obj40:head 2026-02-11T23:31:17.385+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b8510467:::obj40:head v 107'40 2026-02-11T23:31:17.385+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.385+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.385+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.385+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b841f58a:::obj361:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.385+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b841f58a:::obj361:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 112/112 lb 1:b841f58a:::obj361:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.385+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.385+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b841f58a:::obj361:head) v3 2026-02-11T23:31:17.385+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b75b9722:::obj495:head local-lis/les=109/110 n=442 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b841f58a:::obj361:head) 2026-02-11T23:31:17.385+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.385+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b841f58a:::obj361:head) v3 finish latency 0.000011 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.385+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.385+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.385+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x49b9980 con 0x3752000 2026-02-11T23:31:17.385+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b8510467:::obj40:head)]) -- 0x4f51b00 con 0x3752000 2026-02-11T23:31:17.385+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.386+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 926 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d380 con 0x3752000 2026-02-11T23:31:17.386+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.386+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.386+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 927 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b8510467:::obj40:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3752000 2026-02-11T23:31:17.386+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b8510467:::obj40:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.386+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b8510467:::obj40:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 112/109 [PushOp(1:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b8510467:::obj40:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b8f63b78:::obj291:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b8f63b78:::obj291:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b8f63b78:::obj291:head 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b8f63b78:::obj291:head v 107'291 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.386+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b8510467:::obj40:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b8510467:::obj40:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 112/112 lb 1:b8510467:::obj40:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b8510467:::obj40:head) v3 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b841f58a:::obj361:head local-lis/les=109/110 n=443 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b8510467:::obj40:head) 2026-02-11T23:31:17.386+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b8510467:::obj40:head) v3 finish latency 0.000012 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.386+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4a05980 con 0x3752000 2026-02-11T23:31:17.386+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b8f63b78:::obj291:head)]) -- 0x4a05b00 con 0x3752000 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.386+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 928 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d860 con 0x3752000 2026-02-11T23:31:17.386+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.386+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.386+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 929 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b8f63b78:::obj291:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d700 con 0x3752000 2026-02-11T23:31:17.386+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b8f63b78:::obj291:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 112 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.386+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b8f63b78:::obj291:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 112/109 [PushOp(1:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b8f63b78:::obj291:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b92d6f18:::obj505:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b92d6f18:::obj505:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b92d6f18:::obj505:head 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b92d6f18:::obj505:head v 107'505 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.386+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.386+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b8f63b78:::obj291:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b8f63b78:::obj291:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 112/112 lb 1:b8f63b78:::obj291:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b8f63b78:::obj291:head) v3 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8510467:::obj40:head local-lis/les=109/110 n=444 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b8f63b78:::obj291:head) 2026-02-11T23:31:17.386+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b8f63b78:::obj291:head) v3 finish latency 0.000015 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.386+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.386+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.386+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.386+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4a05200 con 0x3752000 2026-02-11T23:31:17.386+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b92d6f18:::obj505:head)]) -- 0x49b9c80 con 0x3752000 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.387+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 930 ==== MOSDPGPush(1.0 112/109 [PushOp(1:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516dd40 con 0x3752000 2026-02-11T23:31:17.387+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 112 2026-02-11T23:31:17.387+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.387+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 931 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b92d6f18:::obj505:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3752000 2026-02-11T23:31:17.387+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b92d6f18:::obj505:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.387+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b92d6f18:::obj505:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b92d6f18:::obj505:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:b9836a99:::obj18:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:b9836a99:::obj18:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:b9836a99:::obj18:head 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:b9836a99:::obj18:head v 107'18 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.387+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b92d6f18:::obj505:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.387+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b92d6f18:::obj505:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 112/112 lb 1:b92d6f18:::obj505:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.387+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.387+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b92d6f18:::obj505:head) v3 2026-02-11T23:31:17.387+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b8f63b78:::obj291:head local-lis/les=109/110 n=445 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b92d6f18:::obj505:head) 2026-02-11T23:31:17.387+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.387+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b92d6f18:::obj505:head) v3 finish latency 0.000024 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.387+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 112 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 112 get_nextmap_reserved map_reservations: {112=1} 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 112 release_map epoch: 112 2026-02-11T23:31:17.387+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4a05500 con 0x3752000 2026-02-11T23:31:17.387+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:b9836a99:::obj18:head)]) -- 0x4f2d800 con 0x3752000 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.387+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 57 ==== osd_map(113..113 src has 1..113) ==== 296+0+0 (crc 0 0 0) 0x46eba40 con 0x396bc00 2026-02-11T23:31:17.387+0000 7fac626e8640 20 osd.0 112 OSD::ms_dispatch: osd_map(113..113 src has 1..113) 2026-02-11T23:31:17.387+0000 7fac626e8640 20 osd.0 112 _dispatch 0x46eba40 osd_map(113..113 src has 1..113) 2026-02-11T23:31:17.387+0000 7fac626e8640 3 osd.0 112 handle_osd_map epochs [113,113], i have 112, src has [1,113] 2026-02-11T23:31:17.387+0000 7fac626e8640 10 osd.0 112 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.387+0000 7fac626e8640 10 osd.0 112 handle_osd_map got inc map for epoch 113 2026-02-11T23:31:17.387+0000 7fac626e8640 20 osd.0 112 got_full_map 113, nothing requested 2026-02-11T23:31:17.387+0000 7fac626e8640 20 osd.0 112 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:17.387+0000 7fac626e8640 20 osd.0 112 handle_osd_map pg_num_history pg_num_history(e113 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:31:17.387+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {113={}} 2026-02-11T23:31:17.387+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:17.387+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 932 ==== MOSDPGPush(1.0 112/109 [PushOp(1:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3752000 2026-02-11T23:31:17.387+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e113 maps [1~113] lci=[5,113] tlb=1) 2026-02-11T23:31:17.387+0000 7fac703fa640 15 osd.0 112 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 112 2026-02-11T23:31:17.387+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.387+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 933 ==== pg_backfill(progress 1.0 e 112/112 lb 1:b9836a99:::obj18:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3752000 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.387+0000 7fac703fa640 15 osd.0 112 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:b9836a99:::obj18:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 112 2026-02-11T23:31:17.387+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b9836a99:::obj18:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 112/109 [PushOp(1:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.387+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b9836a99:::obj18:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ba083999:::obj295:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.387+0000 7fac51168640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ba083999:::obj295:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ba083999:::obj295:head 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ba083999:::obj295:head v 107'295 2026-02-11T23:31:17.387+0000 7fac51168640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.387+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 112 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.388+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:b9836a99:::obj18:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b9836a99:::obj18:head) v3 prio 15 cost 0 latency 0.000107 pg_backfill(progress 1.0 e 112/112 lb 1:b9836a99:::obj18:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 112 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 112 is not newer than session's projected_epoch of 112 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:b9836a99:::obj18:head) v3 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b92d6f18:::obj505:head local-lis/les=109/110 n=446 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:b9836a99:::obj18:head) 2026-02-11T23:31:17.388+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 112 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:b9836a99:::obj18:head) v3 finish latency 0.000023 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.388+0000 7fac5e983640 10 osd.0 112 _committed_osd_maps 113..113 2026-02-11T23:31:17.388+0000 7fac5e983640 10 osd.0 112 advance to epoch 113 (<= last 113 <= newest_map 113) 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0 113 consume_map version 113 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0 112 await_reserved_maps epoch:113 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0 112 await_reserved_maps done 0 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0 113 consume_map 113 -- publish done 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x49cc160 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot 4.0 has no pg and waiting_for_split 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x25809a0 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0 113 identify_splits_and_merges 1.0 e112 to e113 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0 113 identify_splits_and_merges 1.0 e112 to e113 pg_nums {24=1} 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0 113 prune_pg_created 2026-02-11T23:31:17.388+0000 7fac5e983640 10 osd.0:0.consume_map 113 (was 112) 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 empty, pruning 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:17.388+0000 7fac5e983640 10 osd.0:1.consume_map 113 (was 112) 2026-02-11T23:31:17.388+0000 7fac5e983640 10 osd.0:2.consume_map 113 (was 112) 2026-02-11T23:31:17.388+0000 7fac5e983640 10 osd.0:3.consume_map 113 (was 112) 2026-02-11T23:31:17.388+0000 7fac5e983640 10 osd.0:4.consume_map 113 (was 112) 2026-02-11T23:31:17.388+0000 7fac5e983640 10 osd.0:5.consume_map 113 (was 112) 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.388+0000 7fac5e983640 10 osd.0:6.consume_map 113 (was 112) 2026-02-11T23:31:17.388+0000 7fac5e983640 10 osd.0:7.consume_map 113 (was 112) 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 112 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.388+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.388+0000 7fac5e983640 10 osd.0 113 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:17.388+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e112/112) -- 0x4a04d80 con 0x3752000 2026-02-11T23:31:17.388+0000 7fac5e983640 15 osd.0 113 enqueue_peering_evt 1.0 epoch_sent: 113 epoch_requested: 113 NullEvt 2026-02-11T23:31:17.388+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 NullEvt) class_id 2 prio 255 cost 10 e113) 2026-02-11T23:31:17.388+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:ba083999:::obj295:head)]) -- 0x4e70d80 con 0x3752000 2026-02-11T23:31:17.388+0000 7fac5e983640 7 osd.0 113 activate_map version 113 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 NullEvt) class_id 2immediate prio 255 cost 10 e113) queued 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 NullEvt) class_id 2immediate prio 255 cost 10 e113) pg 0x4a2a000 2026-02-11T23:31:17.388+0000 7fac55971640 20 osd.0 113 agent_entry empty queue 2026-02-11T23:31:17.388+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_advance_map: 113 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 112 -> 113 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1] -- 1/1 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] state: Started advmap 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 113 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_activate_map: epoch range: 113..113 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 112 while current is 113 2026-02-11T23:31:17.388+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 113 epoch_requested: 113 NullEvt 2026-02-11T23:31:17.388+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.388+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 934 ==== MOSDPGPush(1.0 112/109 [PushOp(1:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182820 con 0x3752000 2026-02-11T23:31:17.388+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 112 2026-02-11T23:31:17.388+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.388+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 935 ==== pg_backfill(progress 1.0 e 112/112 lb 1:ba083999:::obj295:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5b080 con 0x3752000 2026-02-11T23:31:17.388+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.388+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:ba083999:::obj295:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 112 2026-02-11T23:31:17.388+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ba083999:::obj295:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.388+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.388+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.388+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 112/109 [PushOp(1:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 112 -> 113 (shared) 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ba083999:::obj295:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 113 send_incremental_map epoch range: (113, 113] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 113 add_map_inc_bl 113 252 bytes 2026-02-11T23:31:17.388+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(113..113 src has 1..113) -- 0x3941c00 con 0x3752000 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ba3d236d:::obj529:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.388+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ba3d236d:::obj529:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ba3d236d:::obj529:head 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ba3d236d:::obj529:head v 107'529 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.388+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.388+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000141 2026-02-11T23:31:17.388+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ba083999:::obj295:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ba083999:::obj295:head) v3 prio 15 cost 0 latency 0.000168 pg_backfill(progress 1.0 e 112/112 lb 1:ba083999:::obj295:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:ba083999:::obj295:head) v3 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:b9836a99:::obj18:head local-lis/les=109/110 n=447 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:ba083999:::obj295:head) 2026-02-11T23:31:17.388+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.388+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ba083999:::obj295:head) v3 finish latency 0.000017 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.388+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.389+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4a05080 con 0x3752000 2026-02-11T23:31:17.389+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:ba3d236d:::obj529:head)]) -- 0x4f50180 con 0x3752000 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.389+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 41 ==== PGlog(4.0 log log((107'590,107'600], crt=107'600) pi ([106,111] all_participants={0, 4} intervals=[([106,110] acting={4})]) pg_lease(ru 76.337150574s ub 76.337150574s int 16.000000000s) e113/112) ==== 39009+0+0 (unknown 0 0 0) 0x3901000 con 0x3e4a000 2026-02-11T23:31:17.389+0000 7fac703fa640 15 osd.0 113 enqueue_peering_evt 4.0 epoch_sent: 113 epoch_requested: 112 MLogRec from 4 log log((107'590,107'600], crt=107'600) pi ([106,111] all_participants={0, 4} intervals=[([106,110] acting={4})]) pg_lease(ru 76.337150574s ub 76.337150574s int 16.000000000s) +create_info 2026-02-11T23:31:17.389+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 112 MLogRec from 4 log log((107'590,107'600], crt=107'600) pi ([106,111] all_participants={0, 4} intervals=[([106,110] acting={4})]) pg_lease(ru 76.337150574s ub 76.337150574s int 16.000000000s) +create_info) class_id 2 prio 255 cost 10 e113) 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 (new) to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 112 MLogRec from 4 log log((107'590,107'600], crt=107'600) pi ([106,111] all_participants={0, 4} intervals=[([106,110] acting={4})]) pg_lease(ru 76.337150574s ub 76.337150574s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e113) queued 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 112 MLogRec from 4 log log((107'590,107'600], crt=107'600) pi ([106,111] all_participants={0, 4} intervals=[([106,110] acting={4})]) pg_lease(ru 76.337150574s ub 76.337150574s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e113) pg 0 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 no pg, should create on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 112 MLogRec from 4 log log((107'590,107'600], crt=107'600) pi ([106,111] all_participants={0, 4} intervals=[([106,110] acting={4})]) pg_lease(ru 76.337150574s ub 76.337150574s int 16.000000000s) +create_info) class_id 2immediate prio 255 cost 10 e113) 2026-02-11T23:31:17.389+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 936 ==== MOSDPGPush(1.0 112/109 [PushOp(1:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182d00 con 0x3752000 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 113 _make_pg 4.0 2026-02-11T23:31:17.389+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 112/109 [PushOp(1:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 112 2026-02-11T23:31:17.389+0000 7fac55170640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {}] SnapMapper 2026-02-11T23:31:17.389+0000 7fac55170640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {}] update_bits new_bits: 0 2026-02-11T23:31:17.389+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e112) 2026-02-11T23:31:17.389+0000 7fac55170640 20 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {"0000000000000004."}] update_bits prefix updated 2026-02-11T23:31:17.389+0000 7fac55170640 10 snap_mapper [pg_id:0, match:0, mask_bits:0, last_key_checked:, pool:4, shard:-1, shard_prefix: , prefixes: {"0000000000000004."}] reset_prefix_itr::from <0> to ::update_bits 2026-02-11T23:31:17.389+0000 7fac55170640 5 osd.0 pg_epoch: 112 pg[4.0(unlocked)] enter Initial 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[4.0(unlocked)] enter NotTrimming 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0(unlocked)] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( DNE empty local-lis/les=0/0 n=0 ec=0/0 lis/c=0/0 les/c/f=0/0/0 sis=0) [] r=-1 lpr=0 crt=0'0 unknown mbc={}] PeeringState::init init role -1 up [4,0] acting [4] history ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112 past_intervals ([106,111] all_participants={0, 4} intervals=[([106,110] acting={4})]) 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4]+[4,0] 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::init_hb_stamps now [] 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 70.224472046s + 0.000000000s) 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] cancel_recovery 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] clear_recovery_state 2026-02-11T23:31:17.389+0000 7fac55170640 15 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] _clear_recovery_state 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:17.389+0000 7fac55170640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:17.389+0000 7fac55170640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:17.389+0000 7fac55170640 6 write_log_and_missing with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:31:17.389+0000 7fac55170640 10 _write_log_and_missing clearing up to 0'0 dirty_to_dups=0'0 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:31:17.389+0000 7fac55170640 10 _write_log_and_missing going to encode log.dups.size()=0 2026-02-11T23:31:17.389+0000 7fac55170640 10 _write_log_and_missing 1st round encoded log.dups.size()=0 2026-02-11T23:31:17.389+0000 7fac55170640 10 _write_log_and_missing 2st round encoded log.dups.size()=0 2026-02-11T23:31:17.389+0000 7fac55170640 10 end of _write_log_and_missing 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] handle_initialize 2026-02-11T23:31:17.389+0000 7fac55170640 5 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] exit Initial 0.000077 0 0.000000 2026-02-11T23:31:17.389+0000 7fac55170640 5 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] enter Reset 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=0 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 112 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 112..112 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.389+0000 7fac55170640 5 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] exit Reset 0.000014 1 0.000014 2026-02-11T23:31:17.389+0000 7fac55170640 5 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] enter Started 2026-02-11T23:31:17.389+0000 7fac55170640 5 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] enter Start 2026-02-11T23:31:17.389+0000 7fac55170640 1 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] state: transitioning to Stray 2026-02-11T23:31:17.389+0000 7fac55170640 5 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:31:17.389+0000 7fac55170640 5 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] enter Started/Stray 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 112 while current is 112 2026-02-11T23:31:17.389+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 113 handle_pg_create_info new pg pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0:0._attach_pg 4.0 0x4859000 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 113 identify_splits_and_merges 4.0 e112 to e113 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 113 identify_splits_and_merges 4.0 e112 to e113 pg_nums {45=1} 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0:0._prime_splits 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] handle_advance_map: 113 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 112 -> 113 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 112 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4] -- 4/4 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] state: Started advmap 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 113 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] handle_activate_map: epoch range: 113..113 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.389+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 938 ==== pg_backfill(progress 1.0 e 112/112 lb 1:ba3d236d:::obj529:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3752000 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) queued 2026-02-11T23:31:17.389+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 937 ==== osd_map(113..113 src has 1..113) ==== 296+0+0 (unknown 0 0 0) 0x3941c00 con 0x3752000 2026-02-11T23:31:17.389+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 112/112 lb 1:ba3d236d:::obj529:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 112 2026-02-11T23:31:17.389+0000 7fac606e4640 20 osd.0 113 OSD::ms_dispatch: osd_map(113..113 src has 1..113) 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 112 while current is 113 2026-02-11T23:31:17.389+0000 7fac606e4640 20 osd.0 113 _dispatch 0x3941c00 osd_map(113..113 src has 1..113) 2026-02-11T23:31:17.389+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ba3d236d:::obj529:head) v3) class_id 0 prio 15 cost 0 e112) 2026-02-11T23:31:17.389+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] do_peering_event: epoch_sent: 113 epoch_requested: 112 MLogRec from 4 log log((107'590,107'600], crt=107'600) pi ([106,111] all_participants={0, 4} intervals=[([106,110] acting={4})]) pg_lease(ru 76.337150574s ub 76.337150574s int 16.000000000s) +create_info 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 112/109 [PushOp(1:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e112) pg 0x4a2a000 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( empty local-lis/les=0/0 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=0'0 unknown mbc={}] state: got info+log from osd.4 4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) log((107'590,107'600], crt=107'600) 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000209 MOSDPGPush(1.0 112/109 [PushOp(1:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.389+0000 7fac606e4640 3 osd.0 113 handle_osd_map epochs [113,113], i have 113, src has [1,113] 2026-02-11T23:31:17.389+0000 7fac606e4640 10 osd.0 113 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.389+0000 7fac606e4640 10 osd.0 113 no new maps here, dropping 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 112/109 [PushOp(1:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ba98db39:::obj542:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ba98db39:::obj542:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ba98db39:::obj542:head 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ba98db39:::obj542:head v 107'542 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.389+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 112/109 [PushOp(1:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ba3d236d:::obj529:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) queued 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 112/112 lb 1:ba3d236d:::obj529:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e112) pg 0x4a2a000 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ba3d236d:::obj529:head) v3 prio 15 cost 0 latency 0.000099 pg_backfill(progress 1.0 e 112/112 lb 1:ba3d236d:::obj529:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 112/112 lb 1:ba3d236d:::obj529:head) v3 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba083999:::obj295:head local-lis/les=109/110 n=448 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 112/112 lb 1:ba3d236d:::obj529:head) 2026-02-11T23:31:17.389+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.389+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 112/112 lb 1:ba3d236d:::obj529:head) v3 finish latency 0.000014 2026-02-11T23:31:17.389+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.389+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] PeeringState::proc_lease no-op, !nonprimary 2026-02-11T23:31:17.389+0000 7fac55170640 5 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] exit Started/Stray 0.000222 3 0.000015 2026-02-11T23:31:17.389+0000 7fac55170640 5 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] enter Started/ReplicaActive 2026-02-11T23:31:17.389+0000 7fac55170640 5 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 107'600 -> 107'600 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:31:17.389+0000 7fac55170640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:31:17.389+0000 7fac55170640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:31:17.389+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] state: Activate Finished 2026-02-11T23:31:17.389+0000 7fac55170640 6 write_log_and_missing with: dirty_to: 4294967295'18446744073709551615, dirty_from: 4294967295'18446744073709551615, writeout_from: 4294967295'18446744073709551615, trimmed: , trimmed_dups: , clear_divergent_priors: 0 2026-02-11T23:31:17.389+0000 7fac55170640 10 _write_log_and_missing clearing up to 4294967295'18446744073709551615 dirty_to_dups=4294967295'18446744073709551615 dirty_from_dups=4294967295'18446744073709551615 write_from_dups=4294967295'18446744073709551615 trimmed_dups.size()=0 2026-02-11T23:31:17.389+0000 7fac55170640 10 _write_log_and_missing remove dups min=dup_0000000000.00000000000000000000 to dirty_to_dup=dup_4294967295.18446744073709551615 2026-02-11T23:31:17.389+0000 7fac55170640 10 _write_log_and_missing going to encode log.dups.size()=590 2026-02-11T23:31:17.390+0000 7fac55170640 10 _write_log_and_missing 1st round encoded log.dups.size()=590 2026-02-11T23:31:17.390+0000 7fac55170640 10 _write_log_and_missing 2st round encoded log.dups.size()=590 2026-02-11T23:31:17.390+0000 7fac55170640 10 end of _write_log_and_missing 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.390+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.390+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49a3c80 con 0x3752000 2026-02-11T23:31:17.390+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 112/109 [PushReplyOp(1:ba98db39:::obj542:head)]) -- 0x4f50300 con 0x3752000 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.390+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.390+0000 7fac55170640 15 osd.0 113 enqueue_peering_evt 4.0 epoch_sent: 113 epoch_requested: 113 ActivateCommitted from 113 processed at 113 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 ActivateCommitted from 113 processed at 113) class_id 2 prio 255 cost 10 e113) 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 ActivateCommitted from 113 processed at 113) class_id 2immediate prio 255 cost 10 e113) queued 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 ActivateCommitted from 113 processed at 113) class_id 2immediate prio 255 cost 10 e113) pg 0x4859000 2026-02-11T23:31:17.390+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] do_peering_event: epoch_sent: 113 epoch_requested: 113 ActivateCommitted from 113 processed at 113 2026-02-11T23:31:17.390+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.390+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 crt=107'600 lcod 0'0 unknown mbc={}] state: react 113 telling primary 2026-02-11T23:31:17.390+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.390+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 112 -> 113 (shared) 2026-02-11T23:31:17.390+0000 7fac55170640 10 osd.0 113 send_incremental_map epoch range: (113, 113] to v2:127.0.0.1:6836/864727417 2026-02-11T23:31:17.390+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(113..113 src has 1..113) -- 0x3940c40 con 0x3e4a000 2026-02-11T23:31:17.390+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_info2(4.0 4.0 4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) e113/113) -- 0x5175800 con 0x3e4a000 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.390+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 42 ==== pg_info2(4.0 4.0 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) e113/113) ==== 1111+0+0 (unknown 0 0 0) 0x5175800 con 0x3e4a000 2026-02-11T23:31:17.390+0000 7fac703fa640 15 osd.0 113 enqueue_peering_evt 4.0 epoch_sent: 113 epoch_requested: 113 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) pg_lease(ru 76.338790894s ub 76.338790894s int 16.000000000s) 2026-02-11T23:31:17.390+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) pg_lease(ru 76.338790894s ub 76.338790894s int 16.000000000s)) class_id 2 prio 255 cost 10 e113) 2026-02-11T23:31:17.390+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.390+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) pg_lease(ru 76.338790894s ub 76.338790894s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e113) queued 2026-02-11T23:31:17.390+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.390+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) pg_lease(ru 76.338790894s ub 76.338790894s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e113) pg 0x4859000 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.390+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=106/106 les/c/f=107/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 113 epoch_requested: 113 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) pg_lease(ru 76.338790894s ub 76.338790894s int 16.000000000s) 2026-02-11T23:31:17.390+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] PeeringState::update_history advanced history from ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112 2026-02-11T23:31:17.390+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.390+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 939 ==== MOSDPGPush(1.0 113/109 [PushOp(1:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51831e0 con 0x3752000 2026-02-11T23:31:17.390+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 113 2026-02-11T23:31:17.390+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.390+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.390+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 940 ==== pg_backfill(progress 1.0 e 113/113 lb 1:ba98db39:::obj542:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5b600 con 0x3752000 2026-02-11T23:31:17.390+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:ba98db39:::obj542:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.390+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ba98db39:::obj542:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.390+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.390+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 113/109 [PushOp(1:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.390+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.390+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.390+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ba98db39:::obj542:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.390+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:bb038e88:::obj54:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:bb038e88:::obj54:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:bb038e88:::obj54:head 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:bb038e88:::obj54:head v 107'54 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.391+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.391+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 43 ==== MBackfillReserve(4.0 REQUEST prio: 151 e113/113) ==== 46+0+0 (unknown 0 0 0) 0x4f50300 con 0x3e4a000 2026-02-11T23:31:17.391+0000 7fac703fa640 15 osd.0 113 enqueue_peering_evt 4.0 epoch_sent: 113 epoch_requested: 113 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0 2026-02-11T23:31:17.391+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2 prio 255 cost 10 e113) 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000108 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2immediate prio 255 cost 10 e113) queued 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0) class_id 2immediate prio 255 cost 10 e113) pg 0x4859000 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 113 epoch_requested: 113 RequestBackfillPrio: priority 151 primary bytes 2457600 local bytes 0 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ba98db39:::obj542:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] try_reserve_recovery_space primary_bytes 2400KiB local 0KiB pending_adjustments 2400KiB 2026-02-11T23:31:17.391+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ba98db39:::obj542:head) v3 prio 15 cost 0 latency 0.000138 pg_backfill(progress 1.0 e 113/113 lb 1:ba98db39:::obj542:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] _tentative_full type backfillfull adjust_used 2400KiB 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 compute_adjusted_ratio Before kb_used() 0 2026-02-11T23:31:17.391+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.391+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:ba98db39:::obj542:head) v3 2026-02-11T23:31:17.391+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba3d236d:::obj529:head local-lis/les=109/110 n=449 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:ba98db39:::obj542:head) 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 compute_adjusted_ratio After kb_used() 2400 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0(unlocked)] pg_stat_adjust reserved_num_bytes 596KiB Before kb_used 2400KiB 2026-02-11T23:31:17.391+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0(unlocked)] pg_stat_adjust After kb_used 2996KiB 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 compute_adjusted_ratio backfill adjusted osd_stat(store_statfs(0x97000/0x0/0x384000, data 0x7eba7/0x11c000, compress 0x0/0x0/0x0, omap 0xaba4, meta 0x1a4545c), peers [1,2,3,4] op hist []) 2026-02-11T23:31:17.391+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ba98db39:::obj542:head) v3 finish latency 0.000024 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.391+0000 7fac55170640 5 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.001205 3 0.000186 2026-02-11T23:31:17.391+0000 7fac55170640 5 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] enter Started/ReplicaActive/RepWaitBackfillReserved 2026-02-11T23:31:17.391+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.391+0000 7fac58176640 15 osd.0 113 enqueue_peering_evt 4.0 epoch_sent: 113 epoch_requested: 113 RemoteBackfillReserved 2026-02-11T23:31:17.391+0000 7fac58176640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RemoteBackfillReserved) class_id 2 prio 255 cost 10 e113) 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RemoteBackfillReserved) class_id 2immediate prio 255 cost 10 e113) queued 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RemoteBackfillReserved) class_id 2immediate prio 255 cost 10 e113) pg 0x4859000 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] do_peering_event: epoch_sent: 113 epoch_requested: 113 RemoteBackfillReserved 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.391+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MBackfillReserve(4.0 GRANT e113/113) -- 0x49a3b00 con 0x3e4a000 2026-02-11T23:31:17.391+0000 7fac55170640 5 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] exit Started/ReplicaActive/RepWaitBackfillReserved 0.000036 1 0.000033 2026-02-11T23:31:17.391+0000 7fac55170640 5 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] enter Started/ReplicaActive/RepRecovering 2026-02-11T23:31:17.391+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.391+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49a3380 con 0x3752000 2026-02-11T23:31:17.391+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:bb038e88:::obj54:head)]) -- 0x49a3800 con 0x3752000 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.391+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 941 ==== MOSDPGPush(1.0 113/109 [PushOp(1:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3752000 2026-02-11T23:31:17.391+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 113 2026-02-11T23:31:17.391+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.391+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 942 ==== pg_backfill(progress 1.0 e 113/113 lb 1:bb038e88:::obj54:head) ==== 1061+0+0 (unknown 0 0 0) 0x5162000 con 0x3752000 2026-02-11T23:31:17.391+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:bb038e88:::obj54:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.391+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bb038e88:::obj54:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 113/109 [PushOp(1:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bb038e88:::obj54:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:bbf67374:::obj187:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:bbf67374:::obj187:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:bbf67374:::obj187:head 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:bbf67374:::obj187:head v 107'187 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.391+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bb038e88:::obj54:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.391+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bb038e88:::obj54:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 1.0 e 113/113 lb 1:bb038e88:::obj54:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.391+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.391+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:bb038e88:::obj54:head) v3 2026-02-11T23:31:17.391+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ba98db39:::obj542:head local-lis/les=109/110 n=450 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:bb038e88:::obj54:head) 2026-02-11T23:31:17.391+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.391+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bb038e88:::obj54:head) v3 finish latency 0.000013 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.391+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.391+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.391+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49a3200 con 0x3752000 2026-02-11T23:31:17.391+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:bbf67374:::obj187:head)]) -- 0x49a3680 con 0x3752000 2026-02-11T23:31:17.391+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.392+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 943 ==== MOSDPGPush(1.0 113/109 [PushOp(1:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3752000 2026-02-11T23:31:17.392+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.392+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.392+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 944 ==== pg_backfill(progress 1.0 e 113/113 lb 1:bbf67374:::obj187:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162580 con 0x3752000 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.392+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:bbf67374:::obj187:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.392+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bbf67374:::obj187:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 113/109 [PushOp(1:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bbf67374:::obj187:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:bc55eba4:::obj534:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:bc55eba4:::obj534:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:bc55eba4:::obj534:head 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:bc55eba4:::obj534:head v 107'534 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.392+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bbf67374:::obj187:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.392+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bbf67374:::obj187:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 1.0 e 113/113 lb 1:bbf67374:::obj187:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.392+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.392+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:bbf67374:::obj187:head) v3 2026-02-11T23:31:17.392+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bb038e88:::obj54:head local-lis/les=109/110 n=451 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:bbf67374:::obj187:head) 2026-02-11T23:31:17.392+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.392+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bbf67374:::obj187:head) v3 finish latency 0.000028 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.392+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49b8180 con 0x3752000 2026-02-11T23:31:17.392+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:bc55eba4:::obj534:head)]) -- 0x49b8300 con 0x3752000 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.392+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 945 ==== MOSDPGPush(1.0 113/109 [PushOp(1:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a21a0 con 0x3752000 2026-02-11T23:31:17.392+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 113 2026-02-11T23:31:17.392+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.392+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 946 ==== pg_backfill(progress 1.0 e 113/113 lb 1:bc55eba4:::obj534:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3752000 2026-02-11T23:31:17.392+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:bc55eba4:::obj534:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.392+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bc55eba4:::obj534:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bc55eba4:::obj534:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000052 MOSDPGPush(1.0 113/109 [PushOp(1:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:bd84d997:::obj292:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:bd84d997:::obj292:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:bd84d997:::obj292:head 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:bd84d997:::obj292:head v 107'292 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.392+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.392+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.392+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bc55eba4:::obj534:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.392+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bc55eba4:::obj534:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 113/113 lb 1:bc55eba4:::obj534:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.392+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.392+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:bc55eba4:::obj534:head) v3 2026-02-11T23:31:17.392+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bbf67374:::obj187:head local-lis/les=109/110 n=452 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:bc55eba4:::obj534:head) 2026-02-11T23:31:17.392+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.392+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bc55eba4:::obj534:head) v3 finish latency 0.000016 2026-02-11T23:31:17.392+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.393+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.393+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49a2d80 con 0x3752000 2026-02-11T23:31:17.393+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:bd84d997:::obj292:head)]) -- 0x49a2f00 con 0x3752000 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.393+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 45 ==== pg_scan(get_digest 4.0 MIN-MIN e 113/112) ==== 119+0+0 (unknown 0 0 0) 0x38eac80 con 0x3e4a000 2026-02-11T23:31:17.393+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_scan(get_digest 4.0 MIN-MIN e 113/112) v2 prio 15 type 94 cost 0 latency 0.000006 epoch 113 2026-02-11T23:31:17.393+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 44 ==== osd_map(113..113 src has 1..113) ==== 296+0+0 (unknown 0 0 0) 0x3940c40 con 0x3e4a000 2026-02-11T23:31:17.393+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_scan(get_digest 4.0 MIN-MIN e 113/112) v2) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.393+0000 7fac606e4640 20 osd.0 113 OSD::ms_dispatch: osd_map(113..113 src has 1..113) 2026-02-11T23:31:17.393+0000 7fac606e4640 20 osd.0 113 _dispatch 0x3940c40 osd_map(113..113 src has 1..113) 2026-02-11T23:31:17.393+0000 7fac606e4640 3 osd.0 113 handle_osd_map epochs [113,113], i have 113, src has [1,113] 2026-02-11T23:31:17.393+0000 7fac606e4640 10 osd.0 113 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.393+0000 7fac606e4640 10 osd.0 113 no new maps here, dropping 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_scan(get_digest 4.0 MIN-MIN e 113/112) v2) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_scan(get_digest 4.0 MIN-MIN e 113/112) v2) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 113 dequeue_op pg_scan(get_digest 4.0 MIN-MIN e 113/112) v2 prio 15 cost 0 latency 0.000031 pg_scan(get_digest 4.0 MIN-MIN e 113/112) v2 pg pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] _handle_message: pg_scan(get_digest 4.0 MIN-MIN e 113/112) v2 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] do_scan pg_scan(get_digest 4.0 MIN-MIN e 113/112) 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] scan_range_replica from MIN 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] got 0 items, next MAX 2026-02-11T23:31:17.393+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 947 ==== MOSDPGPush(1.0 113/109 [PushOp(1:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2820 con 0x3752000 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] [] 2026-02-11T23:31:17.393+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.393+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.393+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_scan(digest 4.0 MIN-MAX e 113/112) -- 0x38eaf00 con 0x3e4a000 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 113 dequeue_op pg_scan(get_digest 4.0 MIN-MIN e 113/112) v2 finish latency 0.000062 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.393+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 948 ==== pg_backfill(progress 1.0 e 113/113 lb 1:bd84d997:::obj292:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163080 con 0x3752000 2026-02-11T23:31:17.393+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:bd84d997:::obj292:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.393+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bd84d997:::obj292:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bd84d997:::obj292:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.393+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.393+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.393+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.393+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.393+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:bdc5b82e:::obj52:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:bdc5b82e:::obj52:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.393+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:bdc5b82e:::obj52:head 2026-02-11T23:31:17.393+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:bdc5b82e:::obj52:head v 107'52 2026-02-11T23:31:17.393+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.393+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.393+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bd84d997:::obj292:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bd84d997:::obj292:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 113/113 lb 1:bd84d997:::obj292:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:bd84d997:::obj292:head) v3 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bc55eba4:::obj534:head local-lis/les=109/110 n=453 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:bd84d997:::obj292:head) 2026-02-11T23:31:17.393+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bd84d997:::obj292:head) v3 finish latency 0.000022 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.393+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.393+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49a2900 con 0x3752000 2026-02-11T23:31:17.393+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:bdc5b82e:::obj52:head)]) -- 0x4955e00 con 0x3752000 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.393+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 46 ==== MOSDPGPush(4.0 113/112 [PushOp(4:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3e4a000 2026-02-11T23:31:17.393+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.393+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(4.0 113/112 [PushOp(4:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] handle_push ObjectRecoveryInfo(4:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] submit_push_data 4:006bea6a:::obj172:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] submit_push_data recovering object 4:006bea6a:::obj172:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] on_local_recover: 4:006bea6a:::obj172:head 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] PeeringState::recover_got got missing 4:006bea6a:::obj172:head v 107'172 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.393+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.393+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:006bea6a:::obj172:head, version: 107'172, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:006bea6a:::obj172:head@107'172, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000081 2026-02-11T23:31:17.393+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.393+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 lcod 0'0 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4955b00 con 0x3e4a000 2026-02-11T23:31:17.394+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:006bea6a:::obj172:head)]) -- 0x4d96180 con 0x3e4a000 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 949 ==== MOSDPGPush(1.0 113/109 [PushOp(1:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dfe1a0 con 0x3752000 2026-02-11T23:31:17.394+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.394+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 950 ==== pg_backfill(progress 1.0 e 113/113 lb 1:bdc5b82e:::obj52:head) ==== 1061+0+0 (unknown 0 0 0) 0x5163600 con 0x3752000 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.394+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:bdc5b82e:::obj52:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 113 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bdc5b82e:::obj52:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 113/109 [PushOp(1:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bdc5b82e:::obj52:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:bdf2c185:::obj359:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:bdf2c185:::obj359:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:bdf2c185:::obj359:head 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:bdf2c185:::obj359:head v 107'359 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.394+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bdc5b82e:::obj52:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bdc5b82e:::obj52:head) v3 prio 15 cost 0 latency 0.000100 pg_backfill(progress 1.0 e 113/113 lb 1:bdc5b82e:::obj52:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:bdc5b82e:::obj52:head) v3 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bd84d997:::obj292:head local-lis/les=109/110 n=454 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:bdc5b82e:::obj52:head) 2026-02-11T23:31:17.394+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bdc5b82e:::obj52:head) v3 finish latency 0.000017 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49a2780 con 0x3752000 2026-02-11T23:31:17.394+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:bdf2c185:::obj359:head)]) -- 0x4fe5b00 con 0x3752000 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 47 ==== MOSDPGPush(4.0 113/112 [PushOp(4:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a3520 con 0x3e4a000 2026-02-11T23:31:17.394+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.394+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 48 ==== pg_backfill(progress 4.0 e 113/113 lb 4:006bea6a:::obj172:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163b80 con 0x3e4a000 2026-02-11T23:31:17.394+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:006bea6a:::obj172:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.394+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:006bea6a:::obj172:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 113/112 [PushOp(4:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:006bea6a:::obj172:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:00833431:::obj76:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:00833431:::obj76:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:00833431:::obj76:head 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:00833431:::obj76:head v 107'76 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.394+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:00833431:::obj76:head, version: 107'76, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00833431:::obj76:head@107'76, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:006bea6a:::obj172:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:006bea6a:::obj172:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 113/113 lb 4:006bea6a:::obj172:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:006bea6a:::obj172:head) v3 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb MIN local-lis/les=112/113 n=0 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:006bea6a:::obj172:head) 2026-02-11T23:31:17.394+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:006bea6a:::obj172:head) v3 finish latency 0.000024 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.394+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49a2600 con 0x3e4a000 2026-02-11T23:31:17.394+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:00833431:::obj76:head)]) -- 0x49a2480 con 0x3e4a000 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 951 ==== MOSDPGPush(1.0 113/109 [PushOp(1:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3752000 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 113 2026-02-11T23:31:17.394+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 952 ==== pg_backfill(progress 1.0 e 113/113 lb 1:bdf2c185:::obj359:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3752000 2026-02-11T23:31:17.394+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:bdf2c185:::obj359:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.394+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bdf2c185:::obj359:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 113/109 [PushOp(1:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bdf2c185:::obj359:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:be10f7c9:::obj475:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:be10f7c9:::obj475:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:be10f7c9:::obj475:head 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:be10f7c9:::obj475:head v 107'475 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.394+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.394+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000080 2026-02-11T23:31:17.394+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bdf2c185:::obj359:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bdf2c185:::obj359:head) v3 prio 15 cost 0 latency 0.000105 pg_backfill(progress 1.0 e 113/113 lb 1:bdf2c185:::obj359:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:bdf2c185:::obj359:head) v3 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdc5b82e:::obj52:head local-lis/les=109/110 n=455 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:bdf2c185:::obj359:head) 2026-02-11T23:31:17.394+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.394+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bdf2c185:::obj359:head) v3 finish latency 0.000016 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.394+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.395+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4955800 con 0x3752000 2026-02-11T23:31:17.395+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:be10f7c9:::obj475:head)]) -- 0x4e70480 con 0x3752000 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.395+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 49 ==== MOSDPGPush(4.0 113/112 [PushOp(4:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183d40 con 0x3e4a000 2026-02-11T23:31:17.395+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 113 2026-02-11T23:31:17.395+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.395+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 50 ==== pg_backfill(progress 4.0 e 113/113 lb 4:00833431:::obj76:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.395+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:00833431:::obj76:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.395+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:00833431:::obj76:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 113/112 [PushOp(4:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:00833431:::obj76:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:00b31b6c:::obj19:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:00b31b6c:::obj19:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:00b31b6c:::obj19:head 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:00b31b6c:::obj19:head v 107'19 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.395+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:00b31b6c:::obj19:head, version: 107'19, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:00b31b6c:::obj19:head@107'19, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000079 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:00833431:::obj76:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:00833431:::obj76:head) v3 prio 15 cost 0 latency 0.000105 pg_backfill(progress 4.0 e 113/113 lb 4:00833431:::obj76:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:00833431:::obj76:head) v3 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:006bea6a:::obj172:head local-lis/les=112/113 n=1 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:00833431:::obj76:head) 2026-02-11T23:31:17.395+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:00833431:::obj76:head) v3 finish latency 0.000014 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.395+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 953 ==== MOSDPGPush(1.0 113/109 [PushOp(1:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3752000 2026-02-11T23:31:17.395+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.395+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.395+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 954 ==== pg_backfill(progress 1.0 e 113/113 lb 1:be10f7c9:::obj475:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3752000 2026-02-11T23:31:17.395+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:be10f7c9:::obj475:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.395+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:be10f7c9:::obj475:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 113/109 [PushOp(1:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:be10f7c9:::obj475:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:be1f0c3b:::obj423:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:be1f0c3b:::obj423:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:be1f0c3b:::obj423:head 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:be1f0c3b:::obj423:head v 107'423 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.395+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.395+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:be10f7c9:::obj475:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:be10f7c9:::obj475:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 113/113 lb 1:be10f7c9:::obj475:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:be10f7c9:::obj475:head) v3 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bdf2c185:::obj359:head local-lis/les=109/110 n=456 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:be10f7c9:::obj475:head) 2026-02-11T23:31:17.395+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:be10f7c9:::obj475:head) v3 finish latency 0.000013 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.395+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49a2300 con 0x3e4a000 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.395+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:00b31b6c:::obj19:head)]) -- 0x4fa0d80 con 0x3e4a000 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.395+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.395+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.395+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497fe00 con 0x3752000 2026-02-11T23:31:17.395+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:be1f0c3b:::obj423:head)]) -- 0x4f50900 con 0x3752000 2026-02-11T23:31:17.395+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 955 ==== MOSDPGPush(1.0 113/109 [PushOp(1:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3752000 2026-02-11T23:31:17.396+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.396+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 956 ==== pg_backfill(progress 1.0 e 113/113 lb 1:be1f0c3b:::obj423:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3752000 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.396+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:be1f0c3b:::obj423:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.396+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:be1f0c3b:::obj423:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 113/109 [PushOp(1:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:be1f0c3b:::obj423:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:be604f9b:::obj512:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:be604f9b:::obj512:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:be604f9b:::obj512:head 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:be604f9b:::obj512:head v 107'512 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.396+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:be1f0c3b:::obj423:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:be1f0c3b:::obj423:head) v3 prio 15 cost 0 latency 0.000104 pg_backfill(progress 1.0 e 113/113 lb 1:be1f0c3b:::obj423:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:be1f0c3b:::obj423:head) v3 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be10f7c9:::obj475:head local-lis/les=109/110 n=457 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:be1f0c3b:::obj423:head) 2026-02-11T23:31:17.396+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:be1f0c3b:::obj423:head) v3 finish latency 0.000015 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 51 ==== MOSDPGPush(4.0 113/112 [PushOp(4:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.396+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.396+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 52 ==== pg_backfill(progress 4.0 e 113/113 lb 4:00b31b6c:::obj19:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3e4a000 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.396+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:00b31b6c:::obj19:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 113 2026-02-11T23:31:17.396+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:00b31b6c:::obj19:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 113/112 [PushOp(4:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:00b31b6c:::obj19:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:010e1e72:::obj351:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:010e1e72:::obj351:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:010e1e72:::obj351:head 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:010e1e72:::obj351:head v 107'351 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.396+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:010e1e72:::obj351:head, version: 107'351, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:010e1e72:::obj351:head@107'351, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:00b31b6c:::obj19:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:00b31b6c:::obj19:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 4.0 e 113/113 lb 4:00b31b6c:::obj19:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:00b31b6c:::obj19:head) v3 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00833431:::obj76:head local-lis/les=112/113 n=2 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:00b31b6c:::obj19:head) 2026-02-11T23:31:17.396+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:00b31b6c:::obj19:head) v3 finish latency 0.000014 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.396+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49a2180 con 0x3752000 2026-02-11T23:31:17.396+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:be604f9b:::obj512:head)]) -- 0x4f52180 con 0x3752000 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.396+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x497fb00 con 0x3e4a000 2026-02-11T23:31:17.396+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:010e1e72:::obj351:head)]) -- 0x4f50d80 con 0x3e4a000 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 957 ==== MOSDPGPush(1.0 113/109 [PushOp(1:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3752000 2026-02-11T23:31:17.396+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.396+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 958 ==== pg_backfill(progress 1.0 e 113/113 lb 1:be604f9b:::obj512:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3752000 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.396+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:be604f9b:::obj512:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 113 2026-02-11T23:31:17.396+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:be604f9b:::obj512:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 113/109 [PushOp(1:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:be604f9b:::obj512:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:beac82f5:::obj51:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:beac82f5:::obj51:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:beac82f5:::obj51:head 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:beac82f5:::obj51:head v 107'51 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.396+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.396+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:be604f9b:::obj512:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:be604f9b:::obj512:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 1.0 e 113/113 lb 1:be604f9b:::obj512:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:be604f9b:::obj512:head) v3 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be1f0c3b:::obj423:head local-lis/les=109/110 n=458 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:be604f9b:::obj512:head) 2026-02-11T23:31:17.396+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:be604f9b:::obj512:head) v3 finish latency 0.000015 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.396+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.396+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497fc80 con 0x3752000 2026-02-11T23:31:17.396+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:beac82f5:::obj51:head)]) -- 0x4e70a80 con 0x3752000 2026-02-11T23:31:17.396+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.397+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 53 ==== MOSDPGPush(4.0 113/112 [PushOp(4:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3e4a000 2026-02-11T23:31:17.397+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.397+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.397+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 54 ==== pg_backfill(progress 4.0 e 113/113 lb 4:010e1e72:::obj351:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3e4a000 2026-02-11T23:31:17.397+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:010e1e72:::obj351:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.397+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:010e1e72:::obj351:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:010e1e72:::obj351:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:015e5cd2:::obj354:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:015e5cd2:::obj354:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:015e5cd2:::obj354:head 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:015e5cd2:::obj354:head v 107'354 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.397+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:015e5cd2:::obj354:head, version: 107'354, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:015e5cd2:::obj354:head@107'354, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:010e1e72:::obj351:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:010e1e72:::obj351:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 113/113 lb 4:010e1e72:::obj351:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:010e1e72:::obj351:head) v3 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:00b31b6c:::obj19:head local-lis/les=112/113 n=3 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:010e1e72:::obj351:head) 2026-02-11T23:31:17.397+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:010e1e72:::obj351:head) v3 finish latency 0.000014 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.397+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 959 ==== MOSDPGPush(1.0 113/109 [PushOp(1:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51821a0 con 0x3752000 2026-02-11T23:31:17.397+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.397+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.397+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 960 ==== pg_backfill(progress 1.0 e 113/113 lb 1:beac82f5:::obj51:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3752000 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.397+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:beac82f5:::obj51:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.397+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:beac82f5:::obj51:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:beac82f5:::obj51:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 113/109 [PushOp(1:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:bec6553a:::obj65:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:bec6553a:::obj65:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:bec6553a:::obj65:head 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:bec6553a:::obj65:head v 107'65 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.397+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:beac82f5:::obj51:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:beac82f5:::obj51:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 1.0 e 113/113 lb 1:beac82f5:::obj51:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:beac82f5:::obj51:head) v3 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:be604f9b:::obj512:head local-lis/les=109/110 n=459 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:beac82f5:::obj51:head) 2026-02-11T23:31:17.397+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:beac82f5:::obj51:head) v3 finish latency 0.000014 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.397+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x497f680 con 0x3e4a000 2026-02-11T23:31:17.397+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:015e5cd2:::obj354:head)]) -- 0x4fe5c80 con 0x3e4a000 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.397+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.397+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497f980 con 0x3752000 2026-02-11T23:31:17.397+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:bec6553a:::obj65:head)]) -- 0x4f51980 con 0x3752000 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.397+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 961 ==== MOSDPGPush(1.0 113/109 [PushOp(1:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3752000 2026-02-11T23:31:17.397+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.397+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.397+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 962 ==== pg_backfill(progress 1.0 e 113/113 lb 1:bec6553a:::obj65:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3752000 2026-02-11T23:31:17.397+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:bec6553a:::obj65:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 113 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.397+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bec6553a:::obj65:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 113/109 [PushOp(1:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.397+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bec6553a:::obj65:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.397+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 55 ==== MOSDPGPush(4.0 113/112 [PushOp(4:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d6c0 con 0x3e4a000 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.397+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.397+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.397+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 56 ==== pg_backfill(progress 4.0 e 113/113 lb 4:015e5cd2:::obj354:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f180 con 0x3e4a000 2026-02-11T23:31:17.397+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:015e5cd2:::obj354:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.397+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:015e5cd2:::obj354:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:bf41e297:::obj573:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.397+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:bf41e297:::obj573:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:bf41e297:::obj573:head 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:bf41e297:::obj573:head v 107'573 2026-02-11T23:31:17.397+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.397+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bec6553a:::obj65:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000091 MOSDPGPush(4.0 113/112 [PushOp(4:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bec6553a:::obj65:head) v3 prio 15 cost 0 latency 0.000112 pg_backfill(progress 1.0 e 113/113 lb 1:bec6553a:::obj65:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:bec6553a:::obj65:head) v3 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:beac82f5:::obj51:head local-lis/les=109/110 n=460 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:bec6553a:::obj65:head) 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.398+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:022ea2cc:::obj392:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bec6553a:::obj65:head) v3 finish latency 0.000015 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:022ea2cc:::obj392:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:015e5cd2:::obj354:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:022ea2cc:::obj392:head 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:022ea2cc:::obj392:head v 107'392 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.398+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:022ea2cc:::obj392:head, version: 107'392, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:022ea2cc:::obj392:head@107'392, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:015e5cd2:::obj354:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:015e5cd2:::obj354:head) v3 prio 15 cost 0 latency 0.000137 pg_backfill(progress 4.0 e 113/113 lb 4:015e5cd2:::obj354:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:015e5cd2:::obj354:head) v3 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:010e1e72:::obj351:head local-lis/les=112/113 n=4 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:015e5cd2:::obj354:head) 2026-02-11T23:31:17.398+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:015e5cd2:::obj354:head) v3 finish latency 0.000014 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.398+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497f200 con 0x3752000 2026-02-11T23:31:17.398+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:bf41e297:::obj573:head)]) -- 0x4fa0a80 con 0x3752000 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.398+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x497f500 con 0x3e4a000 2026-02-11T23:31:17.398+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:022ea2cc:::obj392:head)]) -- 0x4fe4c00 con 0x3e4a000 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 963 ==== MOSDPGPush(1.0 113/109 [PushOp(1:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3752000 2026-02-11T23:31:17.398+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.398+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 964 ==== pg_backfill(progress 1.0 e 113/113 lb 1:bf41e297:::obj573:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3752000 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.398+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:bf41e297:::obj573:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 113 2026-02-11T23:31:17.398+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bf41e297:::obj573:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bf41e297:::obj573:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:bf770068:::obj268:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:bf770068:::obj268:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:bf770068:::obj268:head 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:bf770068:::obj268:head v 107'268 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.398+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bf41e297:::obj573:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bf41e297:::obj573:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 1.0 e 113/113 lb 1:bf41e297:::obj573:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:bf41e297:::obj573:head) v3 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bec6553a:::obj65:head local-lis/les=109/110 n=461 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:bf41e297:::obj573:head) 2026-02-11T23:31:17.398+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bf41e297:::obj573:head) v3 finish latency 0.000014 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.398+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4955500 con 0x3752000 2026-02-11T23:31:17.398+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:bf770068:::obj268:head)]) -- 0x49a3380 con 0x3752000 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 57 ==== MOSDPGPush(4.0 113/112 [PushOp(4:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cd00 con 0x3e4a000 2026-02-11T23:31:17.398+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.398+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 58 ==== pg_backfill(progress 4.0 e 113/113 lb 4:022ea2cc:::obj392:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.398+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:022ea2cc:::obj392:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 113 2026-02-11T23:31:17.398+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:022ea2cc:::obj392:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:022ea2cc:::obj392:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:02455f6c:::obj462:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:02455f6c:::obj462:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:02455f6c:::obj462:head 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:02455f6c:::obj462:head v 107'462 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.398+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.398+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:02455f6c:::obj462:head, version: 107'462, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02455f6c:::obj462:head@107'462, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.398+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:022ea2cc:::obj392:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:022ea2cc:::obj392:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 113/113 lb 4:022ea2cc:::obj392:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:022ea2cc:::obj392:head) v3 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:015e5cd2:::obj354:head local-lis/les=112/113 n=5 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:022ea2cc:::obj392:head) 2026-02-11T23:31:17.398+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.398+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:022ea2cc:::obj392:head) v3 finish latency 0.000013 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.398+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.398+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 965 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d040 con 0x3752000 2026-02-11T23:31:17.398+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.398+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c0848ef4:::obj545:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c0848ef4:::obj545:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c0848ef4:::obj545:head 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c0848ef4:::obj545:head v 107'545 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.399+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.399+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x497f080 con 0x3e4a000 2026-02-11T23:31:17.399+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 966 ==== pg_backfill(progress 1.0 e 113/113 lb 1:bf770068:::obj268:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3752000 2026-02-11T23:31:17.399+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:bf770068:::obj268:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.399+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:02455f6c:::obj462:head)]) -- 0x4f50300 con 0x3e4a000 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.399+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bf770068:::obj268:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bf770068:::obj268:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:bf770068:::obj268:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bf770068:::obj268:head) v3 prio 15 cost 0 latency 0.000023 pg_backfill(progress 1.0 e 113/113 lb 1:bf770068:::obj268:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:bf770068:::obj268:head) v3 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf41e297:::obj573:head local-lis/les=109/110 n=462 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:bf770068:::obj268:head) 2026-02-11T23:31:17.399+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:bf770068:::obj268:head) v3 finish latency 0.000024 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.399+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497ec00 con 0x3752000 2026-02-11T23:31:17.399+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c0848ef4:::obj545:head)]) -- 0x4a05080 con 0x3752000 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.399+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 59 ==== MOSDPGPush(4.0 113/112 [PushOp(4:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.399+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.399+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 60 ==== pg_backfill(progress 4.0 e 113/113 lb 4:02455f6c:::obj462:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.399+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:02455f6c:::obj462:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.399+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:02455f6c:::obj462:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 113/112 [PushOp(4:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:02455f6c:::obj462:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:024707f1:::obj490:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:024707f1:::obj490:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:024707f1:::obj490:head 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:024707f1:::obj490:head v 107'490 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.399+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:024707f1:::obj490:head, version: 107'490, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:024707f1:::obj490:head@107'490, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:02455f6c:::obj462:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:02455f6c:::obj462:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 113/113 lb 4:02455f6c:::obj462:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:02455f6c:::obj462:head) v3 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:022ea2cc:::obj392:head local-lis/les=112/113 n=6 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:02455f6c:::obj462:head) 2026-02-11T23:31:17.399+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:02455f6c:::obj462:head) v3 finish latency 0.000014 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.399+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 967 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3752000 2026-02-11T23:31:17.399+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.399+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 968 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c0848ef4:::obj545:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3752000 2026-02-11T23:31:17.399+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c0848ef4:::obj545:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.399+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c0848ef4:::obj545:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c0848ef4:::obj545:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c0c49def:::obj22:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c0c49def:::obj22:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c0c49def:::obj22:head 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c0c49def:::obj22:head v 107'22 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.399+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.399+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c0848ef4:::obj545:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c0848ef4:::obj545:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 113/113 lb 1:c0848ef4:::obj545:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c0848ef4:::obj545:head) v3 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:bf770068:::obj268:head local-lis/les=109/110 n=463 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c0848ef4:::obj545:head) 2026-02-11T23:31:17.399+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c0848ef4:::obj545:head) v3 finish latency 0.000013 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.399+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x497f500 con 0x3e4a000 2026-02-11T23:31:17.399+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:024707f1:::obj490:head)]) -- 0x4a04d80 con 0x3e4a000 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.399+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.399+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.399+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fa0a80 con 0x3752000 2026-02-11T23:31:17.399+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c0c49def:::obj22:head)]) -- 0x4a05500 con 0x3752000 2026-02-11T23:31:17.399+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.400+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 969 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3752000 2026-02-11T23:31:17.400+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.400+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.400+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 970 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c0c49def:::obj22:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5080 con 0x3752000 2026-02-11T23:31:17.400+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c0c49def:::obj22:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.400+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c0c49def:::obj22:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 113/109 [PushOp(1:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c0c49def:::obj22:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.400+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 61 ==== MOSDPGPush(4.0 113/112 [PushOp(4:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.400+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000005 epoch 113 2026-02-11T23:31:17.400+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c138e0af:::obj576:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c138e0af:::obj576:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.400+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 62 ==== pg_backfill(progress 4.0 e 113/113 lb 4:024707f1:::obj490:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5600 con 0x3e4a000 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c138e0af:::obj576:head 2026-02-11T23:31:17.400+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:024707f1:::obj490:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c138e0af:::obj576:head v 107'576 2026-02-11T23:31:17.400+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:024707f1:::obj490:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.400+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c0c49def:::obj22:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000075 MOSDPGPush(4.0 113/112 [PushOp(4:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c0c49def:::obj22:head) v3 prio 15 cost 0 latency 0.000109 pg_backfill(progress 1.0 e 113/113 lb 1:c0c49def:::obj22:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c0c49def:::obj22:head) v3 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0848ef4:::obj545:head local-lis/les=109/110 n=464 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c0c49def:::obj22:head) 2026-02-11T23:31:17.400+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:02817247:::obj101:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:02817247:::obj101:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:02817247:::obj101:head 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c0c49def:::obj22:head) v3 finish latency 0.000016 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:02817247:::obj101:head v 107'101 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.400+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:024707f1:::obj490:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.400+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:02817247:::obj101:head, version: 107'101, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:02817247:::obj101:head@107'101, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:024707f1:::obj490:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:024707f1:::obj490:head) v3 prio 15 cost 0 latency 0.000128 pg_backfill(progress 4.0 e 113/113 lb 4:024707f1:::obj490:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:024707f1:::obj490:head) v3 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02455f6c:::obj462:head local-lis/les=112/113 n=7 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:024707f1:::obj490:head) 2026-02-11T23:31:17.400+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:024707f1:::obj490:head) v3 finish latency 0.000013 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.400+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497f200 con 0x3752000 2026-02-11T23:31:17.400+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c138e0af:::obj576:head)]) -- 0x49b9c80 con 0x3752000 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.400+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.400+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4f51980 con 0x3e4a000 2026-02-11T23:31:17.400+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:02817247:::obj101:head)]) -- 0x4a05b00 con 0x3e4a000 2026-02-11T23:31:17.400+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.400+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 971 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3752000 2026-02-11T23:31:17.400+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.400+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.400+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 972 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c138e0af:::obj576:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3752000 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.400+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c138e0af:::obj576:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 113 2026-02-11T23:31:17.400+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c138e0af:::obj576:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.400+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c138e0af:::obj576:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c19e5add:::obj152:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c19e5add:::obj152:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c19e5add:::obj152:head 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c19e5add:::obj152:head v 107'152 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.401+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c138e0af:::obj576:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c138e0af:::obj576:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 113/113 lb 1:c138e0af:::obj576:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c138e0af:::obj576:head) v3 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c0c49def:::obj22:head local-lis/les=109/110 n=465 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c138e0af:::obj576:head) 2026-02-11T23:31:17.401+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c138e0af:::obj576:head) v3 finish latency 0.000014 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 63 ==== MOSDPGPush(4.0 113/112 [PushOp(4:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.401+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.401+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 64 ==== pg_backfill(progress 4.0 e 113/113 lb 4:02817247:::obj101:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.401+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:02817247:::obj101:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 113 2026-02-11T23:31:17.401+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:02817247:::obj101:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:02817247:::obj101:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:03553543:::obj227:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:03553543:::obj227:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:03553543:::obj227:head 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:03553543:::obj227:head v 107'227 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.401+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:03553543:::obj227:head, version: 107'227, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03553543:::obj227:head@107'227, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:02817247:::obj101:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:02817247:::obj101:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 113/113 lb 4:02817247:::obj101:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:02817247:::obj101:head) v3 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:024707f1:::obj490:head local-lis/les=112/113 n=8 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:02817247:::obj101:head) 2026-02-11T23:31:17.401+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:02817247:::obj101:head) v3 finish latency 0.000014 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.401+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497f980 con 0x3752000 2026-02-11T23:31:17.401+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c19e5add:::obj152:head)]) -- 0x4f51b00 con 0x3752000 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.401+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4fe5c80 con 0x3e4a000 2026-02-11T23:31:17.401+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:03553543:::obj227:head)]) -- 0x4f6e600 con 0x3e4a000 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 973 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3752000 2026-02-11T23:31:17.401+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.401+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 974 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c19e5add:::obj152:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3752000 2026-02-11T23:31:17.401+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c19e5add:::obj152:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 113 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.401+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c19e5add:::obj152:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 113/109 [PushOp(1:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c19e5add:::obj152:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c275c16a:::obj32:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c275c16a:::obj32:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c275c16a:::obj32:head 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c275c16a:::obj32:head v 107'32 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.401+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c19e5add:::obj152:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c19e5add:::obj152:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 113/113 lb 1:c19e5add:::obj152:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c19e5add:::obj152:head) v3 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c138e0af:::obj576:head local-lis/les=109/110 n=466 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c19e5add:::obj152:head) 2026-02-11T23:31:17.401+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c19e5add:::obj152:head) v3 finish latency 0.000014 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.401+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497f680 con 0x3752000 2026-02-11T23:31:17.401+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c275c16a:::obj32:head)]) -- 0x4f6ec00 con 0x3752000 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 65 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.401+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.401+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 66 ==== pg_backfill(progress 4.0 e 113/113 lb 4:03553543:::obj227:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.401+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:03553543:::obj227:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.401+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:03553543:::obj227:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 113/112 [PushOp(4:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:03553543:::obj227:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0366f813:::obj557:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0366f813:::obj557:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0366f813:::obj557:head 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0366f813:::obj557:head v 107'557 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.401+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.401+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0366f813:::obj557:head, version: 107'557, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0366f813:::obj557:head@107'557, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.401+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:03553543:::obj227:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:03553543:::obj227:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 113/113 lb 4:03553543:::obj227:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:03553543:::obj227:head) v3 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:02817247:::obj101:head local-lis/les=112/113 n=9 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:03553543:::obj227:head) 2026-02-11T23:31:17.401+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.401+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:03553543:::obj227:head) v3 finish latency 0.000014 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.401+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.402+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4e70a80 con 0x3e4a000 2026-02-11T23:31:17.402+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0366f813:::obj557:head)]) -- 0x4a05c80 con 0x3e4a000 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.402+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 975 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3752000 2026-02-11T23:31:17.402+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.402+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 976 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c275c16a:::obj32:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f7700 con 0x3752000 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.402+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c275c16a:::obj32:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.402+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c275c16a:::obj32:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c275c16a:::obj32:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c290fff4:::obj320:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c290fff4:::obj320:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c290fff4:::obj320:head 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c290fff4:::obj320:head v 107'320 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.402+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c275c16a:::obj32:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c275c16a:::obj32:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 113/113 lb 1:c275c16a:::obj32:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c275c16a:::obj32:head) v3 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c19e5add:::obj152:head local-lis/les=109/110 n=467 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c275c16a:::obj32:head) 2026-02-11T23:31:17.402+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c275c16a:::obj32:head) v3 finish latency 0.000015 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.402+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497fc80 con 0x3752000 2026-02-11T23:31:17.402+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c290fff4:::obj320:head)]) -- 0x4f2c480 con 0x3752000 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.402+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 67 ==== MOSDPGPush(4.0 113/112 [PushOp(4:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3e4a000 2026-02-11T23:31:17.402+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.402+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 68 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0366f813:::obj557:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164c00 con 0x3e4a000 2026-02-11T23:31:17.402+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0366f813:::obj557:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.402+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0366f813:::obj557:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0366f813:::obj557:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:03b9100b:::obj283:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:03b9100b:::obj283:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:03b9100b:::obj283:head 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:03b9100b:::obj283:head v 107'283 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.402+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.402+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 977 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3752000 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:03b9100b:::obj283:head, version: 107'283, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:03b9100b:::obj283:head@107'283, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.402+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.402+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0366f813:::obj557:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0366f813:::obj557:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 113/113 lb 4:0366f813:::obj557:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0366f813:::obj557:head) v3 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03553543:::obj227:head local-lis/les=112/113 n=10 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0366f813:::obj557:head) 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 978 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c290fff4:::obj320:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165180 con 0x3752000 2026-02-11T23:31:17.402+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.402+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c290fff4:::obj320:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c290fff4:::obj320:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0366f813:::obj557:head) v3 finish latency 0.000016 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c290fff4:::obj320:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 113/109 [PushOp(1:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c3033278:::obj208:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c3033278:::obj208:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c3033278:::obj208:head 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c3033278:::obj208:head v 107'208 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.402+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.402+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.402+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c290fff4:::obj320:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c290fff4:::obj320:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 113/113 lb 1:c290fff4:::obj320:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c290fff4:::obj320:head) v3 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c275c16a:::obj32:head local-lis/les=109/110 n=468 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c290fff4:::obj320:head) 2026-02-11T23:31:17.402+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.402+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c290fff4:::obj320:head) v3 finish latency 0.000014 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.402+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.403+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4f50d80 con 0x3e4a000 2026-02-11T23:31:17.403+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:03b9100b:::obj283:head)]) -- 0x4f2c180 con 0x3e4a000 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.403+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497fb00 con 0x3752000 2026-02-11T23:31:17.403+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c3033278:::obj208:head)]) -- 0x4f2c900 con 0x3752000 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.403+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 979 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3752000 2026-02-11T23:31:17.403+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.403+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.403+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 980 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c3033278:::obj208:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3752000 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.403+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3033278:::obj208:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 113 2026-02-11T23:31:17.403+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3033278:::obj208:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 113/109 [PushOp(1:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3033278:::obj208:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.403+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 69 ==== MOSDPGPush(4.0 113/112 [PushOp(4:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183040 con 0x3e4a000 2026-02-11T23:31:17.403+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c3ae426e:::obj537:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.403+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c3ae426e:::obj537:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.403+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 70 ==== pg_backfill(progress 4.0 e 113/113 lb 4:03b9100b:::obj283:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3e4a000 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c3ae426e:::obj537:head 2026-02-11T23:31:17.403+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:03b9100b:::obj283:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c3ae426e:::obj537:head v 107'537 2026-02-11T23:31:17.403+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:03b9100b:::obj283:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.403+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3033278:::obj208:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000069 MOSDPGPush(4.0 113/112 [PushOp(4:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3033278:::obj208:head) v3 prio 15 cost 0 latency 0.000105 pg_backfill(progress 1.0 e 113/113 lb 1:c3033278:::obj208:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c3033278:::obj208:head) v3 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c290fff4:::obj320:head local-lis/les=109/110 n=469 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c3033278:::obj208:head) 2026-02-11T23:31:17.403+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3033278:::obj208:head) v3 finish latency 0.000016 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:040c73f9:::obj237:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:03b9100b:::obj283:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:040c73f9:::obj237:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:040c73f9:::obj237:head 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:040c73f9:::obj237:head v 107'237 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.403+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.403+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:040c73f9:::obj237:head, version: 107'237, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:040c73f9:::obj237:head@107'237, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:03b9100b:::obj283:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:03b9100b:::obj283:head) v3 prio 15 cost 0 latency 0.000122 pg_backfill(progress 4.0 e 113/113 lb 4:03b9100b:::obj283:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:03b9100b:::obj283:head) v3 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0366f813:::obj557:head local-lis/les=112/113 n=11 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:03b9100b:::obj283:head) 2026-02-11T23:31:17.403+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:03b9100b:::obj283:head) v3 finish latency 0.000014 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.403+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4f52180 con 0x3752000 2026-02-11T23:31:17.403+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c3ae426e:::obj537:head)]) -- 0x4f2ca80 con 0x3752000 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.403+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.403+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49a2180 con 0x3e4a000 2026-02-11T23:31:17.403+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.403+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:040c73f9:::obj237:head)]) -- 0x4f2d200 con 0x3e4a000 2026-02-11T23:31:17.403+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 981 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3752000 2026-02-11T23:31:17.404+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.404+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 982 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c3ae426e:::obj537:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3752000 2026-02-11T23:31:17.404+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3ae426e:::obj537:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 113 2026-02-11T23:31:17.404+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3ae426e:::obj537:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 113/109 [PushOp(1:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3ae426e:::obj537:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c3ba172d:::obj29:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c3ba172d:::obj29:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c3ba172d:::obj29:head 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c3ba172d:::obj29:head v 107'29 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.404+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3ae426e:::obj537:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3ae426e:::obj537:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 1.0 e 113/113 lb 1:c3ae426e:::obj537:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c3ae426e:::obj537:head) v3 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3033278:::obj208:head local-lis/les=109/110 n=470 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c3ae426e:::obj537:head) 2026-02-11T23:31:17.404+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3ae426e:::obj537:head) v3 finish latency 0.000027 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.404+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4f50900 con 0x3752000 2026-02-11T23:31:17.404+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c3ba172d:::obj29:head)]) -- 0x4f2cf00 con 0x3752000 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 71 ==== MOSDPGPush(4.0 113/112 [PushOp(4:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.404+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 113 2026-02-11T23:31:17.404+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 72 ==== pg_backfill(progress 4.0 e 113/113 lb 4:040c73f9:::obj237:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded080 con 0x3e4a000 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.404+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:040c73f9:::obj237:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.404+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:040c73f9:::obj237:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 113/112 [PushOp(4:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:040c73f9:::obj237:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:04510f17:::obj73:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:04510f17:::obj73:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:04510f17:::obj73:head 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:04510f17:::obj73:head v 107'73 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.404+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:04510f17:::obj73:head, version: 107'73, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04510f17:::obj73:head@107'73, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:040c73f9:::obj237:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:040c73f9:::obj237:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 113/113 lb 4:040c73f9:::obj237:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:040c73f9:::obj237:head) v3 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:03b9100b:::obj283:head local-lis/les=112/113 n=12 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:040c73f9:::obj237:head) 2026-02-11T23:31:17.404+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:040c73f9:::obj237:head) v3 finish latency 0.000024 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.404+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x497fe00 con 0x3e4a000 2026-02-11T23:31:17.404+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:04510f17:::obj73:head)]) -- 0x49a2180 con 0x3e4a000 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 983 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d040 con 0x3752000 2026-02-11T23:31:17.404+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.404+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 984 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c3ba172d:::obj29:head) ==== 1061+0+0 (unknown 0 0 0) 0x4ded600 con 0x3752000 2026-02-11T23:31:17.404+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3ba172d:::obj29:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.404+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3ba172d:::obj29:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3ba172d:::obj29:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c3bae056:::obj148:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c3bae056:::obj148:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c3bae056:::obj148:head 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c3bae056:::obj148:head v 107'148 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.404+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.404+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3ba172d:::obj29:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3ba172d:::obj29:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 113/113 lb 1:c3ba172d:::obj29:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c3ba172d:::obj29:head) v3 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ae426e:::obj537:head local-lis/les=109/110 n=471 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c3ba172d:::obj29:head) 2026-02-11T23:31:17.404+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3ba172d:::obj29:head) v3 finish latency 0.000017 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.404+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.404+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fa0d80 con 0x3752000 2026-02-11T23:31:17.404+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c3bae056:::obj148:head)]) -- 0x4f2ca80 con 0x3752000 2026-02-11T23:31:17.404+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.405+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 73 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4445ba0 con 0x3e4a000 2026-02-11T23:31:17.405+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 113 2026-02-11T23:31:17.405+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 74 ==== pg_backfill(progress 4.0 e 113/113 lb 4:04510f17:::obj73:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dee100 con 0x3e4a000 2026-02-11T23:31:17.405+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:04510f17:::obj73:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.405+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:04510f17:::obj73:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(4.0 113/112 [PushOp(4:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:04510f17:::obj73:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0485ab12:::obj244:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0485ab12:::obj244:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0485ab12:::obj244:head 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0485ab12:::obj244:head v 107'244 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.405+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0485ab12:::obj244:head, version: 107'244, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0485ab12:::obj244:head@107'244, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:04510f17:::obj73:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:04510f17:::obj73:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 113/113 lb 4:04510f17:::obj73:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:04510f17:::obj73:head) v3 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:040c73f9:::obj237:head local-lis/les=112/113 n=13 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:04510f17:::obj73:head) 2026-02-11T23:31:17.405+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:04510f17:::obj73:head) v3 finish latency 0.000018 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.405+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 985 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3752000 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.405+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.405+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.405+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49a2300 con 0x3e4a000 2026-02-11T23:31:17.405+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0485ab12:::obj244:head)]) -- 0x4f2c900 con 0x3e4a000 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 986 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c3bae056:::obj148:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3752000 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.405+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3bae056:::obj148:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3bae056:::obj148:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 113/109 [PushOp(1:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3bae056:::obj148:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c3bcd33c:::obj192:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c3bcd33c:::obj192:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c3bcd33c:::obj192:head 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c3bcd33c:::obj192:head v 107'192 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.405+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000079 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3bae056:::obj148:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3bae056:::obj148:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 1.0 e 113/113 lb 1:c3bae056:::obj148:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c3bae056:::obj148:head) v3 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3ba172d:::obj29:head local-lis/les=109/110 n=472 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c3bae056:::obj148:head) 2026-02-11T23:31:17.405+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3bae056:::obj148:head) v3 finish latency 0.000017 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.405+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4e70480 con 0x3752000 2026-02-11T23:31:17.405+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c3bcd33c:::obj192:head)]) -- 0x4f2c180 con 0x3752000 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.405+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 75 ==== MOSDPGPush(4.0 113/112 [PushOp(4:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51821a0 con 0x3e4a000 2026-02-11T23:31:17.405+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 113 2026-02-11T23:31:17.405+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 76 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0485ab12:::obj244:head) ==== 1062+0+0 (unknown 0 0 0) 0x4deec00 con 0x3e4a000 2026-02-11T23:31:17.405+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0485ab12:::obj244:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.405+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0485ab12:::obj244:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 113/112 [PushOp(4:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0485ab12:::obj244:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:04865490:::obj600:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:04865490:::obj600:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:04865490:::obj600:head 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:04865490:::obj600:head v 107'600 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.405+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:04865490:::obj600:head, version: 107'600, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04865490:::obj600:head@107'600, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0485ab12:::obj244:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0485ab12:::obj244:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 113/113 lb 4:0485ab12:::obj244:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0485ab12:::obj244:head) v3 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04510f17:::obj73:head local-lis/les=112/113 n=14 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0485ab12:::obj244:head) 2026-02-11T23:31:17.405+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.405+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0485ab12:::obj244:head) v3 finish latency 0.000016 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.405+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 987 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3752000 2026-02-11T23:31:17.405+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.405+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 988 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c3bcd33c:::obj192:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def180 con 0x3752000 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3bcd33c:::obj192:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.405+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3bcd33c:::obj192:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 113/109 [PushOp(1:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.405+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3bcd33c:::obj192:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c41d6f6c:::obj298:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.405+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c41d6f6c:::obj298:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c41d6f6c:::obj298:head 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c41d6f6c:::obj298:head v 107'298 2026-02-11T23:31:17.405+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.405+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.406+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000085 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c3bcd33c:::obj192:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3bcd33c:::obj192:head) v3 prio 15 cost 0 latency 0.000105 pg_backfill(progress 1.0 e 113/113 lb 1:c3bcd33c:::obj192:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c3bcd33c:::obj192:head) v3 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bae056:::obj148:head local-lis/les=109/110 n=473 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c3bcd33c:::obj192:head) 2026-02-11T23:31:17.406+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c3bcd33c:::obj192:head) v3 finish latency 0.000020 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.406+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4955800 con 0x3e4a000 2026-02-11T23:31:17.406+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:04865490:::obj600:head)]) -- 0x4f2c480 con 0x3e4a000 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.406+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49a2480 con 0x3752000 2026-02-11T23:31:17.406+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c41d6f6c:::obj298:head)]) -- 0x4a05c80 con 0x3752000 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.406+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 989 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x434d040 con 0x3752000 2026-02-11T23:31:17.406+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.406+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.406+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 990 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c41d6f6c:::obj298:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbec00 con 0x3752000 2026-02-11T23:31:17.406+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c41d6f6c:::obj298:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.406+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c41d6f6c:::obj298:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 113/109 [PushOp(1:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c41d6f6c:::obj298:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.406+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 77 ==== MOSDPGPush(4.0 113/112 [PushOp(4:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c4f18846:::obj81:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.406+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c4f18846:::obj81:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.406+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c4f18846:::obj81:head 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c4f18846:::obj81:head v 107'81 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.406+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 78 ==== pg_backfill(progress 4.0 e 113/113 lb 4:04865490:::obj600:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3e4a000 2026-02-11T23:31:17.406+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.406+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:04865490:::obj600:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.406+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:04865490:::obj600:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000079 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c41d6f6c:::obj298:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.406+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c41d6f6c:::obj298:head) v3 prio 15 cost 0 latency 0.000126 pg_backfill(progress 1.0 e 113/113 lb 1:c41d6f6c:::obj298:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.406+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000074 MOSDPGPush(4.0 113/112 [PushOp(4:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.406+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c41d6f6c:::obj298:head) v3 2026-02-11T23:31:17.406+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c3bcd33c:::obj192:head local-lis/les=109/110 n=474 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c41d6f6c:::obj298:head) 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.406+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:048f8fd3:::obj518:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:048f8fd3:::obj518:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:048f8fd3:::obj518:head 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:048f8fd3:::obj518:head v 107'518 2026-02-11T23:31:17.406+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c41d6f6c:::obj298:head) v3 finish latency 0.000029 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.406+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:04865490:::obj600:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:048f8fd3:::obj518:head, version: 107'518, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:048f8fd3:::obj518:head@107'518, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:04865490:::obj600:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.406+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:04865490:::obj600:head) v3 prio 15 cost 0 latency 0.000131 pg_backfill(progress 4.0 e 113/113 lb 4:04865490:::obj600:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.406+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.406+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:04865490:::obj600:head) v3 2026-02-11T23:31:17.406+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0485ab12:::obj244:head local-lis/les=112/113 n=15 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:04865490:::obj600:head) 2026-02-11T23:31:17.406+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.406+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:04865490:::obj600:head) v3 finish latency 0.000018 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.406+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4d96180 con 0x3752000 2026-02-11T23:31:17.406+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c4f18846:::obj81:head)]) -- 0x49a2600 con 0x3752000 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.406+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.406+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.406+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4955b00 con 0x3e4a000 2026-02-11T23:31:17.406+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:048f8fd3:::obj518:head)]) -- 0x4fe5b00 con 0x3e4a000 2026-02-11T23:31:17.406+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.407+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 991 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x4de4340 con 0x3752000 2026-02-11T23:31:17.407+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.407+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.407+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 992 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c4f18846:::obj81:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3752000 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c4f18846:::obj81:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.407+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c4f18846:::obj81:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 113/109 [PushOp(1:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c4f18846:::obj81:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c52c9666:::obj5:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c52c9666:::obj5:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c52c9666:::obj5:head 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c52c9666:::obj5:head v 107'5 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.407+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c4f18846:::obj81:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c4f18846:::obj81:head) v3 prio 15 cost 0 latency 0.000102 pg_backfill(progress 1.0 e 113/113 lb 1:c4f18846:::obj81:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c4f18846:::obj81:head) v3 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c41d6f6c:::obj298:head local-lis/les=109/110 n=475 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c4f18846:::obj81:head) 2026-02-11T23:31:17.407+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c4f18846:::obj81:head) v3 finish latency 0.000021 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.407+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49a2900 con 0x3752000 2026-02-11T23:31:17.407+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c52c9666:::obj5:head)]) -- 0x4955e00 con 0x3752000 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.407+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 79 ==== MOSDPGPush(4.0 113/112 [PushOp(4:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3e4a000 2026-02-11T23:31:17.407+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000005 epoch 113 2026-02-11T23:31:17.407+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 80 ==== pg_backfill(progress 4.0 e 113/113 lb 4:048f8fd3:::obj518:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3e4a000 2026-02-11T23:31:17.407+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:048f8fd3:::obj518:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 113 2026-02-11T23:31:17.407+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:048f8fd3:::obj518:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:048f8fd3:::obj518:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(4.0 113/112 [PushOp(4:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:04b71422:::obj226:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:04b71422:::obj226:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:04b71422:::obj226:head 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:04b71422:::obj226:head v 107'226 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.407+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:04b71422:::obj226:head, version: 107'226, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:04b71422:::obj226:head@107'226, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:048f8fd3:::obj518:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:048f8fd3:::obj518:head) v3 prio 15 cost 0 latency 0.000100 pg_backfill(progress 4.0 e 113/113 lb 4:048f8fd3:::obj518:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:048f8fd3:::obj518:head) v3 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04865490:::obj600:head local-lis/les=112/113 n=16 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:048f8fd3:::obj518:head) 2026-02-11T23:31:17.407+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:048f8fd3:::obj518:head) v3 finish latency 0.000023 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.407+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49a2d80 con 0x3e4a000 2026-02-11T23:31:17.407+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:04b71422:::obj226:head)]) -- 0x4f6ec00 con 0x3e4a000 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.407+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 993 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3752000 2026-02-11T23:31:17.407+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.407+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 994 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c52c9666:::obj5:head) ==== 1060+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3752000 2026-02-11T23:31:17.407+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c52c9666:::obj5:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 113 2026-02-11T23:31:17.407+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c52c9666:::obj5:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c52c9666:::obj5:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c57e020a:::obj395:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c57e020a:::obj395:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c57e020a:::obj395:head 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c57e020a:::obj395:head v 107'395 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.407+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.407+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.407+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c52c9666:::obj5:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c52c9666:::obj5:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 1.0 e 113/113 lb 1:c52c9666:::obj5:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c52c9666:::obj5:head) v3 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c4f18846:::obj81:head local-lis/les=109/110 n=476 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c52c9666:::obj5:head) 2026-02-11T23:31:17.407+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.407+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c52c9666:::obj5:head) v3 finish latency 0.000019 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.407+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.408+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49b8300 con 0x3752000 2026-02-11T23:31:17.408+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c57e020a:::obj395:head)]) -- 0x4f6e600 con 0x3752000 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.408+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 81 ==== MOSDPGPush(4.0 113/112 [PushOp(4:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a21a0 con 0x3e4a000 2026-02-11T23:31:17.408+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.408+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.408+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 82 ==== pg_backfill(progress 4.0 e 113/113 lb 4:04b71422:::obj226:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3e4a000 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.408+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:04b71422:::obj226:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.408+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:04b71422:::obj226:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 113/112 [PushOp(4:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:04b71422:::obj226:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:05651d4b:::obj461:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:05651d4b:::obj461:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:05651d4b:::obj461:head 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:05651d4b:::obj461:head v 107'461 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.408+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:05651d4b:::obj461:head, version: 107'461, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05651d4b:::obj461:head@107'461, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:04b71422:::obj226:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:04b71422:::obj226:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 113/113 lb 4:04b71422:::obj226:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:04b71422:::obj226:head) v3 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:048f8fd3:::obj518:head local-lis/les=112/113 n=17 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:04b71422:::obj226:head) 2026-02-11T23:31:17.408+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:04b71422:::obj226:head) v3 finish latency 0.000020 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.408+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 995 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2680 con 0x3752000 2026-02-11T23:31:17.408+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.408+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.408+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 996 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c57e020a:::obj395:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3752000 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.408+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c57e020a:::obj395:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.408+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c57e020a:::obj395:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c57e020a:::obj395:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 113/109 [PushOp(1:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c59209e1:::obj523:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c59209e1:::obj523:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c59209e1:::obj523:head 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c59209e1:::obj523:head v 107'523 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.408+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.408+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c57e020a:::obj395:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c57e020a:::obj395:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 113/113 lb 1:c57e020a:::obj395:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c57e020a:::obj395:head) v3 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c52c9666:::obj5:head local-lis/les=109/110 n=477 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c57e020a:::obj395:head) 2026-02-11T23:31:17.408+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c57e020a:::obj395:head) v3 finish latency 0.000017 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.408+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49b8180 con 0x3e4a000 2026-02-11T23:31:17.408+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:05651d4b:::obj461:head)]) -- 0x4f51b00 con 0x3e4a000 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.408+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.408+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49a3680 con 0x3752000 2026-02-11T23:31:17.408+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c59209e1:::obj523:head)]) -- 0x4a05b00 con 0x3752000 2026-02-11T23:31:17.408+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.408+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 83 ==== MOSDPGPush(4.0 113/112 [PushOp(4:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2b60 con 0x3e4a000 2026-02-11T23:31:17.408+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.408+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.408+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 84 ==== pg_backfill(progress 4.0 e 113/113 lb 4:05651d4b:::obj461:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3e4a000 2026-02-11T23:31:17.408+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:05651d4b:::obj461:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.408+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.408+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05651d4b:::obj461:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05651d4b:::obj461:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 113/112 [PushOp(4:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:05793669:::obj415:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:05793669:::obj415:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:05793669:::obj415:head 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:05793669:::obj415:head v 107'415 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.409+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.409+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 997 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dfe1a0 con 0x3752000 2026-02-11T23:31:17.409+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.409+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.409+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 998 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c59209e1:::obj523:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3752000 2026-02-11T23:31:17.409+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c59209e1:::obj523:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.409+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c59209e1:::obj523:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:05793669:::obj415:head, version: 107'415, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05793669:::obj415:head@107'415, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05651d4b:::obj461:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 113/109 [PushOp(1:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:05651d4b:::obj461:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 4.0 e 113/113 lb 4:05651d4b:::obj461:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:05651d4b:::obj461:head) v3 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:04b71422:::obj226:head local-lis/les=112/113 n=18 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:05651d4b:::obj461:head) 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c5df6474:::obj459:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c5df6474:::obj459:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.409+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c5df6474:::obj459:head 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c5df6474:::obj459:head v 107'459 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:05651d4b:::obj461:head) v3 finish latency 0.000020 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.409+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c59209e1:::obj523:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c59209e1:::obj523:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c59209e1:::obj523:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 113/113 lb 1:c59209e1:::obj523:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c59209e1:::obj523:head) v3 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c57e020a:::obj395:head local-lis/les=109/110 n=478 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c59209e1:::obj523:head) 2026-02-11T23:31:17.409+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c59209e1:::obj523:head) v3 finish latency 0.000015 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.409+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49a3200 con 0x3e4a000 2026-02-11T23:31:17.409+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:05793669:::obj415:head)]) -- 0x49b9c80 con 0x3e4a000 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.409+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497ef00 con 0x3752000 2026-02-11T23:31:17.409+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c5df6474:::obj459:head)]) -- 0x4a05500 con 0x3752000 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.409+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 85 ==== MOSDPGPush(4.0 113/112 [PushOp(4:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3520 con 0x3e4a000 2026-02-11T23:31:17.409+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.409+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 86 ==== pg_backfill(progress 4.0 e 113/113 lb 4:05793669:::obj415:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5bb80 con 0x3e4a000 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.409+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:05793669:::obj415:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.409+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05793669:::obj415:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 113/112 [PushOp(4:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05793669:::obj415:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:05e4a07c:::obj265:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:05e4a07c:::obj265:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:05e4a07c:::obj265:head 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:05e4a07c:::obj265:head v 107'265 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.409+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.409+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 999 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183d40 con 0x3752000 2026-02-11T23:31:17.409+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.409+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.409+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1000 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c5df6474:::obj459:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3752000 2026-02-11T23:31:17.409+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c5df6474:::obj459:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.409+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c5df6474:::obj459:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:05e4a07c:::obj265:head, version: 107'265, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05e4a07c:::obj265:head@107'265, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 113/109 [PushOp(1:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05793669:::obj415:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:05793669:::obj415:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 4.0 e 113/113 lb 4:05793669:::obj415:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:05793669:::obj415:head) v3 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05651d4b:::obj461:head local-lis/les=112/113 n=19 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:05793669:::obj415:head) 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c5e569a1:::obj444:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.409+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c5e569a1:::obj444:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c5e569a1:::obj444:head 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c5e569a1:::obj444:head v 107'444 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.409+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:05793669:::obj415:head) v3 finish latency 0.000022 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c5df6474:::obj459:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.409+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.409+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c5df6474:::obj459:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c5df6474:::obj459:head) v3 prio 15 cost 0 latency 0.000105 pg_backfill(progress 1.0 e 113/113 lb 1:c5df6474:::obj459:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c5df6474:::obj459:head) v3 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c59209e1:::obj523:head local-lis/les=109/110 n=479 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c5df6474:::obj459:head) 2026-02-11T23:31:17.409+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c5df6474:::obj459:head) v3 finish latency 0.000029 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.409+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.409+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x497e780 con 0x3e4a000 2026-02-11T23:31:17.409+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:05e4a07c:::obj265:head)]) -- 0x4a04d80 con 0x3e4a000 2026-02-11T23:31:17.409+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.410+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497ea80 con 0x3752000 2026-02-11T23:31:17.410+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c5e569a1:::obj444:head)]) -- 0x4a05080 con 0x3752000 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.410+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 87 ==== MOSDPGPush(4.0 113/112 [PushOp(4:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3e4a000 2026-02-11T23:31:17.410+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.410+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.410+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 88 ==== pg_backfill(progress 4.0 e 113/113 lb 4:05e4a07c:::obj265:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3e4a000 2026-02-11T23:31:17.410+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:05e4a07c:::obj265:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.410+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05e4a07c:::obj265:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05e4a07c:::obj265:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:05eb3c5d:::obj514:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:05eb3c5d:::obj514:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:05eb3c5d:::obj514:head 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:05eb3c5d:::obj514:head v 107'514 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.410+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.410+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1001 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3752000 2026-02-11T23:31:17.410+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.410+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.410+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1002 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c5e569a1:::obj444:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3752000 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:05eb3c5d:::obj514:head, version: 107'514, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:05eb3c5d:::obj514:head@107'514, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:17.410+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c5e569a1:::obj444:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.410+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c5e569a1:::obj444:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05e4a07c:::obj265:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 113/109 [PushOp(1:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:05e4a07c:::obj265:head) v3 prio 15 cost 0 latency 0.000107 pg_backfill(progress 4.0 e 113/113 lb 4:05e4a07c:::obj265:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:05e4a07c:::obj265:head) v3 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05793669:::obj415:head local-lis/les=112/113 n=20 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:05e4a07c:::obj265:head) 2026-02-11T23:31:17.410+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c64b60bf:::obj563:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c64b60bf:::obj563:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c64b60bf:::obj563:head 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c64b60bf:::obj563:head v 107'563 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:05e4a07c:::obj265:head) v3 finish latency 0.000029 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.410+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c5e569a1:::obj444:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.410+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c5e569a1:::obj444:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c5e569a1:::obj444:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 113/113 lb 1:c5e569a1:::obj444:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c5e569a1:::obj444:head) v3 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5df6474:::obj459:head local-lis/les=109/110 n=480 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c5e569a1:::obj444:head) 2026-02-11T23:31:17.410+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c5e569a1:::obj444:head) v3 finish latency 0.000015 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.410+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4955200 con 0x3e4a000 2026-02-11T23:31:17.410+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:05eb3c5d:::obj514:head)]) -- 0x4f50300 con 0x3e4a000 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.410+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.410+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.410+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497e600 con 0x3752000 2026-02-11T23:31:17.410+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c64b60bf:::obj563:head)]) -- 0x49a3380 con 0x3752000 2026-02-11T23:31:17.410+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.411+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 89 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.411+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.411+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 90 ==== pg_backfill(progress 4.0 e 113/113 lb 4:05eb3c5d:::obj514:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bb080 con 0x3e4a000 2026-02-11T23:31:17.411+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:05eb3c5d:::obj514:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 113 2026-02-11T23:31:17.411+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05eb3c5d:::obj514:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 113/112 [PushOp(4:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05eb3c5d:::obj514:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0698d46e:::obj173:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0698d46e:::obj173:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0698d46e:::obj173:head 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0698d46e:::obj173:head v 107'173 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.411+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.411+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1003 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2680 con 0x3752000 2026-02-11T23:31:17.411+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.411+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.411+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1004 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c64b60bf:::obj563:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd180 con 0x3752000 2026-02-11T23:31:17.411+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c64b60bf:::obj563:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.411+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c64b60bf:::obj563:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0698d46e:::obj173:head, version: 107'173, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0698d46e:::obj173:head@107'173, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000093 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000061 MOSDPGPush(1.0 113/109 [PushOp(1:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:05eb3c5d:::obj514:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:05eb3c5d:::obj514:head) v3 prio 15 cost 0 latency 0.000152 pg_backfill(progress 4.0 e 113/113 lb 4:05eb3c5d:::obj514:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:05eb3c5d:::obj514:head) v3 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05e4a07c:::obj265:head local-lis/les=112/113 n=21 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:05eb3c5d:::obj514:head) 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c68dea80:::obj235:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.411+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c68dea80:::obj235:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c68dea80:::obj235:head 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c68dea80:::obj235:head v 107'235 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:05eb3c5d:::obj514:head) v3 finish latency 0.000022 2026-02-11T23:31:17.411+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c64b60bf:::obj563:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000075 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c64b60bf:::obj563:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c64b60bf:::obj563:head) v3 prio 15 cost 0 latency 0.000131 pg_backfill(progress 1.0 e 113/113 lb 1:c64b60bf:::obj563:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c64b60bf:::obj563:head) v3 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c5e569a1:::obj444:head local-lis/les=109/110 n=481 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c64b60bf:::obj563:head) 2026-02-11T23:31:17.411+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c64b60bf:::obj563:head) v3 finish latency 0.000016 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.411+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x497e180 con 0x3e4a000 2026-02-11T23:31:17.411+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0698d46e:::obj173:head)]) -- 0x4fe4c00 con 0x3e4a000 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.411+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497e480 con 0x3752000 2026-02-11T23:31:17.411+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c68dea80:::obj235:head)]) -- 0x49b9c80 con 0x3752000 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.411+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 91 ==== MOSDPGPush(4.0 113/112 [PushOp(4:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3e4a000 2026-02-11T23:31:17.411+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.411+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 92 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0698d46e:::obj173:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.411+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0698d46e:::obj173:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 113 2026-02-11T23:31:17.411+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0698d46e:::obj173:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 113/112 [PushOp(4:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:06c5d68c:::obj504:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0698d46e:::obj173:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:06c5d68c:::obj504:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:06c5d68c:::obj504:head 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:06c5d68c:::obj504:head v 107'504 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.411+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:06c5d68c:::obj504:head, version: 107'504, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06c5d68c:::obj504:head@107'504, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.411+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1005 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51824e0 con 0x3752000 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000005 epoch 113 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0698d46e:::obj173:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.411+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0698d46e:::obj173:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 113/113 lb 4:0698d46e:::obj173:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.411+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1006 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c68dea80:::obj235:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3752000 2026-02-11T23:31:17.411+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c68dea80:::obj235:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0698d46e:::obj173:head) v3 2026-02-11T23:31:17.411+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c68dea80:::obj235:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:05eb3c5d:::obj514:head local-lis/les=112/113 n=22 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0698d46e:::obj173:head) 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.411+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 113/109 [PushOp(1:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.411+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0698d46e:::obj173:head) v3 finish latency 0.000024 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.411+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c68dea80:::obj235:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c71bdb5a:::obj349:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.411+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c71bdb5a:::obj349:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c71bdb5a:::obj349:head 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c71bdb5a:::obj349:head v 107'349 2026-02-11T23:31:17.411+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.411+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c68dea80:::obj235:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c68dea80:::obj235:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 113/113 lb 1:c68dea80:::obj235:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c68dea80:::obj235:head) v3 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c64b60bf:::obj563:head local-lis/les=109/110 n=482 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c68dea80:::obj235:head) 2026-02-11T23:31:17.412+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c68dea80:::obj235:head) v3 finish latency 0.000017 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.412+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4917c80 con 0x3e4a000 2026-02-11T23:31:17.412+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:06c5d68c:::obj504:head)]) -- 0x4a05b00 con 0x3e4a000 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.412+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497e000 con 0x3752000 2026-02-11T23:31:17.412+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c71bdb5a:::obj349:head)]) -- 0x4f51b00 con 0x3752000 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.412+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1007 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182000 con 0x3752000 2026-02-11T23:31:17.412+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.412+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.412+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1008 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c71bdb5a:::obj349:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5080 con 0x3752000 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.412+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c71bdb5a:::obj349:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 113 2026-02-11T23:31:17.412+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c71bdb5a:::obj349:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(1.0 113/109 [PushOp(1:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c71bdb5a:::obj349:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c7481937:::obj131:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c7481937:::obj131:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c7481937:::obj131:head 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c7481937:::obj131:head v 107'131 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.412+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.412+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 93 ==== MOSDPGPush(4.0 113/112 [PushOp(4:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516da00 con 0x3e4a000 2026-02-11T23:31:17.412+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.412+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.412+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 94 ==== pg_backfill(progress 4.0 e 113/113 lb 4:06c5d68c:::obj504:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.412+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:06c5d68c:::obj504:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.412+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:06c5d68c:::obj504:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c71bdb5a:::obj349:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c71bdb5a:::obj349:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 113/113 lb 1:c71bdb5a:::obj349:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:06d56074:::obj162:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c71bdb5a:::obj349:head) v3 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c68dea80:::obj235:head local-lis/les=109/110 n=483 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c71bdb5a:::obj349:head) 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:06d56074:::obj162:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:06d56074:::obj162:head 2026-02-11T23:31:17.412+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:06d56074:::obj162:head v 107'162 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.412+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c71bdb5a:::obj349:head) v3 finish latency 0.000020 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:06c5d68c:::obj504:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.412+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:06d56074:::obj162:head, version: 107'162, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:06d56074:::obj162:head@107'162, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:06c5d68c:::obj504:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:06c5d68c:::obj504:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 113/113 lb 4:06c5d68c:::obj504:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:06c5d68c:::obj504:head) v3 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0698d46e:::obj173:head local-lis/les=112/113 n=23 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:06c5d68c:::obj504:head) 2026-02-11T23:31:17.412+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:06c5d68c:::obj504:head) v3 finish latency 0.000018 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.412+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.412+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.412+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.412+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4917800 con 0x3752000 2026-02-11T23:31:17.413+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c7481937:::obj131:head)]) -- 0x4f6e600 con 0x3752000 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.413+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4917b00 con 0x3e4a000 2026-02-11T23:31:17.413+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:06d56074:::obj162:head)]) -- 0x4f6ec00 con 0x3e4a000 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1009 ==== MOSDPGPush(1.0 113/109 [PushOp(1:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182680 con 0x3752000 2026-02-11T23:31:17.413+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.413+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1010 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c7481937:::obj131:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3752000 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.413+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c7481937:::obj131:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 113 2026-02-11T23:31:17.413+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c7481937:::obj131:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(1.0 113/109 [PushOp(1:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c7481937:::obj131:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:c8495ae6:::obj149:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:c8495ae6:::obj149:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:c8495ae6:::obj149:head 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:c8495ae6:::obj149:head v 107'149 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.413+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c7481937:::obj131:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c7481937:::obj131:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 113/113 lb 1:c7481937:::obj131:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c7481937:::obj131:head) v3 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c71bdb5a:::obj349:head local-lis/les=109/110 n=484 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c7481937:::obj131:head) 2026-02-11T23:31:17.413+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c7481937:::obj131:head) v3 finish latency 0.000022 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 95 ==== MOSDPGPush(4.0 113/112 [PushOp(4:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cea0 con 0x3e4a000 2026-02-11T23:31:17.413+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.413+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 96 ==== pg_backfill(progress 4.0 e 113/113 lb 4:06d56074:::obj162:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3e4a000 2026-02-11T23:31:17.413+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:06d56074:::obj162:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 113 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.413+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:06d56074:::obj162:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:06d56074:::obj162:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 113/112 [PushOp(4:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:07c44a4d:::obj181:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:07c44a4d:::obj181:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:07c44a4d:::obj181:head 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:07c44a4d:::obj181:head v 107'181 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.413+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:07c44a4d:::obj181:head, version: 107'181, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:07c44a4d:::obj181:head@107'181, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:06d56074:::obj162:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:06d56074:::obj162:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 113/113 lb 4:06d56074:::obj162:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:06d56074:::obj162:head) v3 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06c5d68c:::obj504:head local-lis/les=112/113 n=24 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:06d56074:::obj162:head) 2026-02-11T23:31:17.413+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:06d56074:::obj162:head) v3 finish latency 0.000018 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4917500 con 0x3752000 2026-02-11T23:31:17.413+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:c8495ae6:::obj149:head)]) -- 0x4955e00 con 0x3752000 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.413+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4917680 con 0x3e4a000 2026-02-11T23:31:17.413+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:07c44a4d:::obj181:head)]) -- 0x4fe5b00 con 0x3e4a000 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1011 ==== MOSDPGPush(1.0 113/109 [PushOp(1:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3752000 2026-02-11T23:31:17.413+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.413+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1012 ==== pg_backfill(progress 1.0 e 113/113 lb 1:c8495ae6:::obj149:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3752000 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.413+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:c8495ae6:::obj149:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 113 2026-02-11T23:31:17.413+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c8495ae6:::obj149:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 113/109 [PushOp(1:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c8495ae6:::obj149:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ca82ed30:::obj135:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ca82ed30:::obj135:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ca82ed30:::obj135:head 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ca82ed30:::obj135:head v 107'135 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.413+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.413+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.413+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.413+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:c8495ae6:::obj149:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.413+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c8495ae6:::obj149:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 113/113 lb 1:c8495ae6:::obj149:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:c8495ae6:::obj149:head) v3 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c7481937:::obj131:head local-lis/les=109/110 n=485 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:c8495ae6:::obj149:head) 2026-02-11T23:31:17.414+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:c8495ae6:::obj149:head) v3 finish latency 0.000021 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.414+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4954f00 con 0x3752000 2026-02-11T23:31:17.414+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:ca82ed30:::obj135:head)]) -- 0x49a2600 con 0x3752000 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.414+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 97 ==== MOSDPGPush(4.0 113/112 [PushOp(4:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.414+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.414+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 98 ==== pg_backfill(progress 4.0 e 113/113 lb 4:07c44a4d:::obj181:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3e4a000 2026-02-11T23:31:17.414+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:07c44a4d:::obj181:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.414+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:07c44a4d:::obj181:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:07c44a4d:::obj181:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:080cd0cf:::obj146:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:080cd0cf:::obj146:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:080cd0cf:::obj146:head 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:080cd0cf:::obj146:head v 107'146 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.414+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:080cd0cf:::obj146:head, version: 107'146, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:080cd0cf:::obj146:head@107'146, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:07c44a4d:::obj181:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:07c44a4d:::obj181:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 113/113 lb 4:07c44a4d:::obj181:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:07c44a4d:::obj181:head) v3 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:06d56074:::obj162:head local-lis/les=112/113 n=25 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:07c44a4d:::obj181:head) 2026-02-11T23:31:17.414+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:07c44a4d:::obj181:head) v3 finish latency 0.000030 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.414+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1013 ==== MOSDPGPush(1.0 113/109 [PushOp(1:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3752000 2026-02-11T23:31:17.414+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.414+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1014 ==== pg_backfill(progress 1.0 e 113/113 lb 1:ca82ed30:::obj135:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3752000 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.414+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:ca82ed30:::obj135:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.414+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ca82ed30:::obj135:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(1.0 113/109 [PushOp(1:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ca82ed30:::obj135:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:cadaa6e3:::obj220:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:cadaa6e3:::obj220:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:cadaa6e3:::obj220:head 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:cadaa6e3:::obj220:head v 107'220 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.414+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ca82ed30:::obj135:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ca82ed30:::obj135:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 113/113 lb 1:ca82ed30:::obj135:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:ca82ed30:::obj135:head) v3 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:c8495ae6:::obj149:head local-lis/les=109/110 n=486 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:ca82ed30:::obj135:head) 2026-02-11T23:31:17.414+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ca82ed30:::obj135:head) v3 finish latency 0.000023 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.414+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4917380 con 0x3e4a000 2026-02-11T23:31:17.414+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:080cd0cf:::obj146:head)]) -- 0x4a05c80 con 0x3e4a000 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.414+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.414+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4954d80 con 0x3752000 2026-02-11T23:31:17.414+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:cadaa6e3:::obj220:head)]) -- 0x4f2c480 con 0x3752000 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.414+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1015 ==== MOSDPGPush(1.0 113/109 [PushOp(1:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3752000 2026-02-11T23:31:17.414+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.414+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1016 ==== pg_backfill(progress 1.0 e 113/113 lb 1:cadaa6e3:::obj220:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3752000 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.414+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:cadaa6e3:::obj220:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 113 2026-02-11T23:31:17.414+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cadaa6e3:::obj220:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 113/109 [PushOp(1:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cadaa6e3:::obj220:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:cb1a5635:::obj233:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:cb1a5635:::obj233:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:cb1a5635:::obj233:head 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:cb1a5635:::obj233:head v 107'233 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.414+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.414+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 99 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3e4a000 2026-02-11T23:31:17.414+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.414+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.414+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 100 ==== pg_backfill(progress 4.0 e 113/113 lb 4:080cd0cf:::obj146:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3e4a000 2026-02-11T23:31:17.414+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:080cd0cf:::obj146:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.414+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:080cd0cf:::obj146:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.414+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cadaa6e3:::obj220:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cadaa6e3:::obj220:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 113/113 lb 1:cadaa6e3:::obj220:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.414+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 113/112 [PushOp(4:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:cadaa6e3:::obj220:head) v3 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ca82ed30:::obj135:head local-lis/les=109/110 n=487 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:cadaa6e3:::obj220:head) 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.415+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0877fdce:::obj530:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0877fdce:::obj530:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0877fdce:::obj530:head 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cadaa6e3:::obj220:head) v3 finish latency 0.000028 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0877fdce:::obj530:head v 107'530 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:080cd0cf:::obj146:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.415+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0877fdce:::obj530:head, version: 107'530, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0877fdce:::obj530:head@107'530, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:080cd0cf:::obj146:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:080cd0cf:::obj146:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 4.0 e 113/113 lb 4:080cd0cf:::obj146:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:080cd0cf:::obj146:head) v3 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:07c44a4d:::obj181:head local-lis/les=112/113 n=26 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:080cd0cf:::obj146:head) 2026-02-11T23:31:17.415+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:080cd0cf:::obj146:head) v3 finish latency 0.000018 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.415+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4917200 con 0x3752000 2026-02-11T23:31:17.415+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:cb1a5635:::obj233:head)]) -- 0x4f2c180 con 0x3752000 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.415+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4954a80 con 0x3e4a000 2026-02-11T23:31:17.415+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0877fdce:::obj530:head)]) -- 0x4f2c900 con 0x3e4a000 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1017 ==== MOSDPGPush(1.0 113/109 [PushOp(1:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3752000 2026-02-11T23:31:17.415+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.415+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.415+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1018 ==== pg_backfill(progress 1.0 e 113/113 lb 1:cb1a5635:::obj233:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3752000 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.415+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb1a5635:::obj233:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.415+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb1a5635:::obj233:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb1a5635:::obj233:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 113/109 [PushOp(1:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:cb1be4ca:::obj406:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:cb1be4ca:::obj406:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:cb1be4ca:::obj406:head 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:cb1be4ca:::obj406:head v 107'406 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.415+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb1a5635:::obj233:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb1a5635:::obj233:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 113/113 lb 1:cb1a5635:::obj233:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:cb1a5635:::obj233:head) v3 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cadaa6e3:::obj220:head local-lis/les=109/110 n=488 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:cb1a5635:::obj233:head) 2026-02-11T23:31:17.415+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb1a5635:::obj233:head) v3 finish latency 0.000021 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.415+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4917080 con 0x3752000 2026-02-11T23:31:17.415+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:cb1be4ca:::obj406:head)]) -- 0x4f2ca80 con 0x3752000 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 101 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.415+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000005 epoch 113 2026-02-11T23:31:17.415+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.415+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 102 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0877fdce:::obj530:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3e4a000 2026-02-11T23:31:17.415+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0877fdce:::obj530:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.415+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0877fdce:::obj530:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0877fdce:::obj530:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 113/112 [PushOp(4:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0881dee6:::obj412:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0881dee6:::obj412:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0881dee6:::obj412:head 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0881dee6:::obj412:head v 107'412 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.415+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.415+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0881dee6:::obj412:head, version: 107'412, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0881dee6:::obj412:head@107'412, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0877fdce:::obj530:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0877fdce:::obj530:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 113/113 lb 4:0877fdce:::obj530:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0877fdce:::obj530:head) v3 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:080cd0cf:::obj146:head local-lis/les=112/113 n=27 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0877fdce:::obj530:head) 2026-02-11T23:31:17.415+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0877fdce:::obj530:head) v3 finish latency 0.000019 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.415+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.415+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4954780 con 0x3e4a000 2026-02-11T23:31:17.415+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0881dee6:::obj412:head)]) -- 0x49a2180 con 0x3e4a000 2026-02-11T23:31:17.415+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1019 ==== MOSDPGPush(1.0 113/109 [PushOp(1:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3752000 2026-02-11T23:31:17.416+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.416+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1020 ==== pg_backfill(progress 1.0 e 113/113 lb 1:cb1be4ca:::obj406:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3752000 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.416+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb1be4ca:::obj406:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.416+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb1be4ca:::obj406:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb1be4ca:::obj406:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 113/109 [PushOp(1:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:cb4c4ca4:::obj480:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:cb4c4ca4:::obj480:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:cb4c4ca4:::obj480:head 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:cb4c4ca4:::obj480:head v 107'480 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.416+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb1be4ca:::obj406:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb1be4ca:::obj406:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 113/113 lb 1:cb1be4ca:::obj406:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:cb1be4ca:::obj406:head) v3 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1a5635:::obj233:head local-lis/les=109/110 n=489 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:cb1be4ca:::obj406:head) 2026-02-11T23:31:17.416+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb1be4ca:::obj406:head) v3 finish latency 0.000020 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 103 ==== MOSDPGPush(4.0 113/112 [PushOp(4:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.416+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.416+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.416+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4916f00 con 0x3752000 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 104 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0881dee6:::obj412:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3e4a000 2026-02-11T23:31:17.416+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:cb4c4ca4:::obj480:head)]) -- 0x4f2cf00 con 0x3752000 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.416+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0881dee6:::obj412:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.416+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0881dee6:::obj412:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 113/112 [PushOp(4:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0881dee6:::obj412:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:08a5234b:::obj62:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:08a5234b:::obj62:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:08a5234b:::obj62:head 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:08a5234b:::obj62:head v 107'62 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.416+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:08a5234b:::obj62:head, version: 107'62, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08a5234b:::obj62:head@107'62, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0881dee6:::obj412:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0881dee6:::obj412:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 113/113 lb 4:0881dee6:::obj412:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0881dee6:::obj412:head) v3 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0877fdce:::obj530:head local-lis/les=112/113 n=28 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0881dee6:::obj412:head) 2026-02-11T23:31:17.416+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0881dee6:::obj412:head) v3 finish latency 0.000024 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.416+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4954480 con 0x3e4a000 2026-02-11T23:31:17.416+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:08a5234b:::obj62:head)]) -- 0x4f2d200 con 0x3e4a000 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1021 ==== MOSDPGPush(1.0 113/109 [PushOp(1:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3752000 2026-02-11T23:31:17.416+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.416+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1022 ==== pg_backfill(progress 1.0 e 113/113 lb 1:cb4c4ca4:::obj480:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d700 con 0x3752000 2026-02-11T23:31:17.416+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb4c4ca4:::obj480:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.416+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb4c4ca4:::obj480:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 113/109 [PushOp(1:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb4c4ca4:::obj480:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:cb843a8e:::obj34:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:cb843a8e:::obj34:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:cb843a8e:::obj34:head 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:cb843a8e:::obj34:head v 107'34 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.416+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.416+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb4c4ca4:::obj480:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb4c4ca4:::obj480:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 113/113 lb 1:cb4c4ca4:::obj480:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:cb4c4ca4:::obj480:head) v3 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb1be4ca:::obj406:head local-lis/les=109/110 n=490 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:cb4c4ca4:::obj480:head) 2026-02-11T23:31:17.416+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb4c4ca4:::obj480:head) v3 finish latency 0.000016 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.416+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.416+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4916d80 con 0x3752000 2026-02-11T23:31:17.416+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:cb843a8e:::obj34:head)]) -- 0x4917200 con 0x3752000 2026-02-11T23:31:17.416+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.417+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 105 ==== MOSDPGPush(4.0 113/112 [PushOp(4:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3e4a000 2026-02-11T23:31:17.417+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.417+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 113/112 [PushOp(4:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.417+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 106 ==== pg_backfill(progress 4.0 e 113/113 lb 4:08a5234b:::obj62:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3e4a000 2026-02-11T23:31:17.417+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:08a5234b:::obj62:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.417+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:08a5234b:::obj62:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:08fb0692:::obj136:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:08fb0692:::obj136:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:08fb0692:::obj136:head 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:08a5234b:::obj62:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:08fb0692:::obj136:head v 107'136 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.417+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:08fb0692:::obj136:head, version: 107'136, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:08fb0692:::obj136:head@107'136, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:08a5234b:::obj62:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:08a5234b:::obj62:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 113/113 lb 4:08a5234b:::obj62:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:08a5234b:::obj62:head) v3 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0881dee6:::obj412:head local-lis/les=112/113 n=29 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:08a5234b:::obj62:head) 2026-02-11T23:31:17.417+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:08a5234b:::obj62:head) v3 finish latency 0.000021 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.417+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1023 ==== MOSDPGPush(1.0 113/109 [PushOp(1:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183a00 con 0x3752000 2026-02-11T23:31:17.417+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.417+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1024 ==== pg_backfill(progress 1.0 e 113/113 lb 1:cb843a8e:::obj34:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3752000 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.417+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb843a8e:::obj34:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.417+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb843a8e:::obj34:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(1.0 113/109 [PushOp(1:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb843a8e:::obj34:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:cbce4a0b:::obj95:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:cbce4a0b:::obj95:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:cbce4a0b:::obj95:head 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:cbce4a0b:::obj95:head v 107'95 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.417+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cb843a8e:::obj34:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb843a8e:::obj34:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 113/113 lb 1:cb843a8e:::obj34:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:cb843a8e:::obj34:head) v3 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb4c4ca4:::obj480:head local-lis/les=109/110 n=491 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:cb843a8e:::obj34:head) 2026-02-11T23:31:17.417+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cb843a8e:::obj34:head) v3 finish latency 0.000016 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.417+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4916900 con 0x3e4a000 2026-02-11T23:31:17.417+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:08fb0692:::obj136:head)]) -- 0x4f2c480 con 0x3e4a000 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.417+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4916c00 con 0x3752000 2026-02-11T23:31:17.417+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:cbce4a0b:::obj95:head)]) -- 0x4a05c80 con 0x3752000 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.417+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1025 ==== MOSDPGPush(1.0 113/109 [PushOp(1:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3752000 2026-02-11T23:31:17.417+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.417+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1026 ==== pg_backfill(progress 1.0 e 113/113 lb 1:cbce4a0b:::obj95:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3752000 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.417+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:cbce4a0b:::obj95:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 113 2026-02-11T23:31:17.417+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cbce4a0b:::obj95:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(1.0 113/109 [PushOp(1:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cbce4a0b:::obj95:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:cc461e5d:::obj45:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:cc461e5d:::obj45:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.417+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 107 ==== MOSDPGPush(4.0 113/112 [PushOp(4:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:cc461e5d:::obj45:head 2026-02-11T23:31:17.417+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000005 epoch 113 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:cc461e5d:::obj45:head v 107'45 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.417+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.417+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.417+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 108 ==== pg_backfill(progress 4.0 e 113/113 lb 4:08fb0692:::obj136:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3e4a000 2026-02-11T23:31:17.417+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:08fb0692:::obj136:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.417+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:08fb0692:::obj136:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cbce4a0b:::obj95:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(4.0 113/112 [PushOp(4:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cbce4a0b:::obj95:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 113/113 lb 1:cbce4a0b:::obj95:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:cbce4a0b:::obj95:head) v3 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cb843a8e:::obj34:head local-lis/les=109/110 n=492 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:cbce4a0b:::obj95:head) 2026-02-11T23:31:17.417+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:095c6b7a:::obj79:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:095c6b7a:::obj79:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:095c6b7a:::obj79:head 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cbce4a0b:::obj95:head) v3 finish latency 0.000021 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:095c6b7a:::obj79:head v 107'79 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:08fb0692:::obj136:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.417+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:095c6b7a:::obj79:head, version: 107'79, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:095c6b7a:::obj79:head@107'79, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.417+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:08fb0692:::obj136:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:08fb0692:::obj136:head) v3 prio 15 cost 0 latency 0.000101 pg_backfill(progress 4.0 e 113/113 lb 4:08fb0692:::obj136:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:08fb0692:::obj136:head) v3 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08a5234b:::obj62:head local-lis/les=112/113 n=30 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:08fb0692:::obj136:head) 2026-02-11T23:31:17.417+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.417+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:08fb0692:::obj136:head) v3 finish latency 0.000019 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.417+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.418+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4916480 con 0x3752000 2026-02-11T23:31:17.418+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:cc461e5d:::obj45:head)]) -- 0x4917380 con 0x3752000 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.418+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4916780 con 0x3e4a000 2026-02-11T23:31:17.418+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:095c6b7a:::obj79:head)]) -- 0x49a2600 con 0x3e4a000 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1027 ==== MOSDPGPush(1.0 113/109 [PushOp(1:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3752000 2026-02-11T23:31:17.418+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.418+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1028 ==== pg_backfill(progress 1.0 e 113/113 lb 1:cc461e5d:::obj45:head) ==== 1061+0+0 (unknown 0 0 0) 0x5162000 con 0x3752000 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.418+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:cc461e5d:::obj45:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.418+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cc461e5d:::obj45:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(1.0 113/109 [PushOp(1:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cc461e5d:::obj45:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:cdb46b38:::obj588:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:cdb46b38:::obj588:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:cdb46b38:::obj588:head 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:cdb46b38:::obj588:head v 107'588 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.418+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cc461e5d:::obj45:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cc461e5d:::obj45:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 113/113 lb 1:cc461e5d:::obj45:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:cc461e5d:::obj45:head) v3 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cbce4a0b:::obj95:head local-lis/les=109/110 n=493 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:cc461e5d:::obj45:head) 2026-02-11T23:31:17.418+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cc461e5d:::obj45:head) v3 finish latency 0.000022 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.418+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4916000 con 0x3752000 2026-02-11T23:31:17.418+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:cdb46b38:::obj588:head)]) -- 0x4954f00 con 0x3752000 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 109 ==== MOSDPGPush(4.0 113/112 [PushOp(4:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.418+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.418+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.418+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 110 ==== pg_backfill(progress 4.0 e 113/113 lb 4:095c6b7a:::obj79:head) ==== 1061+0+0 (unknown 0 0 0) 0x5162580 con 0x3e4a000 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 113/112 [PushOp(4:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.418+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:095c6b7a:::obj79:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.418+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:095c6b7a:::obj79:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:095c6b7a:::obj79:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:097a0595:::obj169:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:097a0595:::obj169:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:097a0595:::obj169:head 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:097a0595:::obj169:head v 107'169 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.418+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:097a0595:::obj169:head, version: 107'169, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:097a0595:::obj169:head@107'169, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:095c6b7a:::obj79:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:095c6b7a:::obj79:head) v3 prio 15 cost 0 latency 0.000054 pg_backfill(progress 4.0 e 113/113 lb 4:095c6b7a:::obj79:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:095c6b7a:::obj79:head) v3 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:08fb0692:::obj136:head local-lis/les=112/113 n=31 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:095c6b7a:::obj79:head) 2026-02-11T23:31:17.418+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:095c6b7a:::obj79:head) v3 finish latency 0.000020 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1029 ==== MOSDPGPush(1.0 113/109 [PushOp(1:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d040 con 0x3752000 2026-02-11T23:31:17.418+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.418+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1030 ==== pg_backfill(progress 1.0 e 113/113 lb 1:cdb46b38:::obj588:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3752000 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.418+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:cdb46b38:::obj588:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.418+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cdb46b38:::obj588:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cdb46b38:::obj588:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 113/109 [PushOp(1:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:cde49336:::obj263:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:cde49336:::obj263:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:cde49336:::obj263:head 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:cde49336:::obj263:head v 107'263 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.418+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.418+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.418+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cdb46b38:::obj588:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cdb46b38:::obj588:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 113/113 lb 1:cdb46b38:::obj588:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:cdb46b38:::obj588:head) v3 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cc461e5d:::obj45:head local-lis/les=109/110 n=494 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:cdb46b38:::obj588:head) 2026-02-11T23:31:17.418+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cdb46b38:::obj588:head) v3 finish latency 0.000016 2026-02-11T23:31:17.418+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.418+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.419+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4916300 con 0x3e4a000 2026-02-11T23:31:17.419+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:097a0595:::obj169:head)]) -- 0x4917680 con 0x3e4a000 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.419+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4954300 con 0x3752000 2026-02-11T23:31:17.419+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:cde49336:::obj263:head)]) -- 0x4917500 con 0x3752000 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.419+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1031 ==== MOSDPGPush(1.0 113/109 [PushOp(1:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4445ba0 con 0x3752000 2026-02-11T23:31:17.419+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.419+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.419+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1032 ==== pg_backfill(progress 1.0 e 113/113 lb 1:cde49336:::obj263:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163080 con 0x3752000 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.419+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:cde49336:::obj263:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.419+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cde49336:::obj263:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cde49336:::obj263:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 113/109 [PushOp(1:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:cf1e3413:::obj224:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:cf1e3413:::obj224:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:cf1e3413:::obj224:head 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:cf1e3413:::obj224:head v 107'224 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.419+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.419+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 111 ==== MOSDPGPush(4.0 113/112 [PushOp(4:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3e4a000 2026-02-11T23:31:17.419+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.419+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.419+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 112 ==== pg_backfill(progress 4.0 e 113/113 lb 4:097a0595:::obj169:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163600 con 0x3e4a000 2026-02-11T23:31:17.419+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:097a0595:::obj169:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.419+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:097a0595:::obj169:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cde49336:::obj263:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cde49336:::obj263:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 113/113 lb 1:cde49336:::obj263:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:cde49336:::obj263:head) v3 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cdb46b38:::obj588:head local-lis/les=109/110 n=495 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:cde49336:::obj263:head) 2026-02-11T23:31:17.419+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(4.0 113/112 [PushOp(4:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cde49336:::obj263:head) v3 finish latency 0.000020 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:097a0595:::obj169:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:09d0cce6:::obj481:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:09d0cce6:::obj481:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:09d0cce6:::obj481:head 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:09d0cce6:::obj481:head v 107'481 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.419+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.419+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:09d0cce6:::obj481:head, version: 107'481, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09d0cce6:::obj481:head@107'481, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:097a0595:::obj169:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:097a0595:::obj169:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 4.0 e 113/113 lb 4:097a0595:::obj169:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:097a0595:::obj169:head) v3 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:095c6b7a:::obj79:head local-lis/les=112/113 n=32 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:097a0595:::obj169:head) 2026-02-11T23:31:17.419+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:097a0595:::obj169:head) v3 finish latency 0.000015 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.419+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4ba3080 con 0x3752000 2026-02-11T23:31:17.419+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:cf1e3413:::obj224:head)]) -- 0x4917b00 con 0x3752000 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.419+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.419+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.419+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4954000 con 0x3e4a000 2026-02-11T23:31:17.419+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:09d0cce6:::obj481:head)]) -- 0x4917800 con 0x3e4a000 2026-02-11T23:31:17.419+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1033 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51821a0 con 0x3752000 2026-02-11T23:31:17.420+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.420+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1034 ==== pg_backfill(progress 1.0 e 113/113 lb 1:cf1e3413:::obj224:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163b80 con 0x3752000 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.420+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:cf1e3413:::obj224:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 113 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cf1e3413:::obj224:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 113/109 [PushOp(1:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d05ea685:::obj543:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d05ea685:::obj543:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d05ea685:::obj543:head 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cf1e3413:::obj224:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d05ea685:::obj543:head v 107'543 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.420+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:cf1e3413:::obj224:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cf1e3413:::obj224:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 113/113 lb 1:cf1e3413:::obj224:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:cf1e3413:::obj224:head) v3 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cde49336:::obj263:head local-lis/les=109/110 n=496 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:cf1e3413:::obj224:head) 2026-02-11T23:31:17.420+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:cf1e3413:::obj224:head) v3 finish latency 0.000022 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.420+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4ba2480 con 0x3752000 2026-02-11T23:31:17.420+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d05ea685:::obj543:head)]) -- 0x497e000 con 0x3752000 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 113 ==== MOSDPGPush(4.0 113/112 [PushOp(4:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3e4a000 2026-02-11T23:31:17.420+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.420+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 114 ==== pg_backfill(progress 4.0 e 113/113 lb 4:09d0cce6:::obj481:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3e4a000 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.420+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:09d0cce6:::obj481:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.420+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:09d0cce6:::obj481:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 113/112 [PushOp(4:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:09d0cce6:::obj481:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:09eb3833:::obj488:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:09eb3833:::obj488:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:09eb3833:::obj488:head 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:09eb3833:::obj488:head v 107'488 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.420+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:09eb3833:::obj488:head, version: 107'488, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:09eb3833:::obj488:head@107'488, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:09d0cce6:::obj481:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:09d0cce6:::obj481:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 113/113 lb 4:09d0cce6:::obj481:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:09d0cce6:::obj481:head) v3 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:097a0595:::obj169:head local-lis/les=112/113 n=33 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:09d0cce6:::obj481:head) 2026-02-11T23:31:17.420+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:09d0cce6:::obj481:head) v3 finish latency 0.000017 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1035 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3752000 2026-02-11T23:31:17.420+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.420+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.420+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4ba3c80 con 0x3e4a000 2026-02-11T23:31:17.420+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:09eb3833:::obj488:head)]) -- 0x4917c80 con 0x3e4a000 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1036 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d05ea685:::obj543:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3752000 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.420+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d05ea685:::obj543:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.420+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d05ea685:::obj543:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d05ea685:::obj543:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d06c69cb:::obj532:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d06c69cb:::obj532:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d06c69cb:::obj532:head 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d06c69cb:::obj532:head v 107'532 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.420+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.420+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d05ea685:::obj543:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d05ea685:::obj543:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 113/113 lb 1:d05ea685:::obj543:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d05ea685:::obj543:head) v3 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:cf1e3413:::obj224:head local-lis/les=109/110 n=497 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d05ea685:::obj543:head) 2026-02-11T23:31:17.420+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d05ea685:::obj543:head) v3 finish latency 0.000017 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.420+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4910600 con 0x3752000 2026-02-11T23:31:17.420+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d06c69cb:::obj532:head)]) -- 0x497e480 con 0x3752000 2026-02-11T23:31:17.420+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.420+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 115 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.420+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.420+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.421+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 116 ==== pg_backfill(progress 4.0 e 113/113 lb 4:09eb3833:::obj488:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.421+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:09eb3833:::obj488:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.421+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:09eb3833:::obj488:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 113/112 [PushOp(4:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:09eb3833:::obj488:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0a4fcbe7:::obj507:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0a4fcbe7:::obj507:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0a4fcbe7:::obj507:head 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0a4fcbe7:::obj507:head v 107'507 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.421+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0a4fcbe7:::obj507:head, version: 107'507, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0a4fcbe7:::obj507:head@107'507, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:09eb3833:::obj488:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:09eb3833:::obj488:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 113/113 lb 4:09eb3833:::obj488:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:09eb3833:::obj488:head) v3 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09d0cce6:::obj481:head local-lis/les=112/113 n=34 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:09eb3833:::obj488:head) 2026-02-11T23:31:17.421+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:09eb3833:::obj488:head) v3 finish latency 0.000029 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1037 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3752000 2026-02-11T23:31:17.421+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.421+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1038 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d06c69cb:::obj532:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3752000 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.421+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d06c69cb:::obj532:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.421+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d06c69cb:::obj532:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d06c69cb:::obj532:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d0e799f1:::obj299:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d0e799f1:::obj299:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d0e799f1:::obj299:head 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d0e799f1:::obj299:head v 107'299 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.421+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d06c69cb:::obj532:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d06c69cb:::obj532:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 1.0 e 113/113 lb 1:d06c69cb:::obj532:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d06c69cb:::obj532:head) v3 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d05ea685:::obj543:head local-lis/les=109/110 n=498 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d06c69cb:::obj532:head) 2026-02-11T23:31:17.421+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d06c69cb:::obj532:head) v3 finish latency 0.000016 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.421+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4911e00 con 0x3e4a000 2026-02-11T23:31:17.421+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0a4fcbe7:::obj507:head)]) -- 0x497e180 con 0x3e4a000 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.421+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4910000 con 0x3752000 2026-02-11T23:31:17.421+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d0e799f1:::obj299:head)]) -- 0x497e600 con 0x3752000 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 117 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.421+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.421+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 118 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0a4fcbe7:::obj507:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd700 con 0x3e4a000 2026-02-11T23:31:17.421+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0a4fcbe7:::obj507:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 113 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.421+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0a4fcbe7:::obj507:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0a4fcbe7:::obj507:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 113/112 [PushOp(4:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0b3f05c8:::obj589:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0b3f05c8:::obj589:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0b3f05c8:::obj589:head 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0b3f05c8:::obj589:head v 107'589 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.421+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.421+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1039 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3752000 2026-02-11T23:31:17.421+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.421+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.421+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1040 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d0e799f1:::obj299:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3752000 2026-02-11T23:31:17.421+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d0e799f1:::obj299:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.421+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d0e799f1:::obj299:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0b3f05c8:::obj589:head, version: 107'589, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0b3f05c8:::obj589:head@107'589, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0a4fcbe7:::obj507:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 113/109 [PushOp(1:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0a4fcbe7:::obj507:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 113/113 lb 4:0a4fcbe7:::obj507:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0a4fcbe7:::obj507:head) v3 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:09eb3833:::obj488:head local-lis/les=112/113 n=35 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0a4fcbe7:::obj507:head) 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.421+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0a4fcbe7:::obj507:head) v3 finish latency 0.000018 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d0f049f1:::obj387:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d0e799f1:::obj299:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d0f049f1:::obj387:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d0f049f1:::obj387:head 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d0f049f1:::obj387:head v 107'387 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.421+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.421+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d0e799f1:::obj299:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d0e799f1:::obj299:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 1.0 e 113/113 lb 1:d0e799f1:::obj299:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d0e799f1:::obj299:head) v3 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d06c69cb:::obj532:head local-lis/les=109/110 n=499 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d0e799f1:::obj299:head) 2026-02-11T23:31:17.421+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d0e799f1:::obj299:head) v3 finish latency 0.000013 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.421+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4910180 con 0x3e4a000 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0b3f05c8:::obj589:head)]) -- 0x4955200 con 0x3e4a000 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.421+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.421+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4eabc80 con 0x3752000 2026-02-11T23:31:17.421+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d0f049f1:::obj387:head)]) -- 0x497ea80 con 0x3752000 2026-02-11T23:31:17.421+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.422+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1041 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3752000 2026-02-11T23:31:17.422+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.422+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1042 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d0f049f1:::obj387:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3752000 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.422+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d0f049f1:::obj387:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.422+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d0f049f1:::obj387:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 113/109 [PushOp(1:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d0f049f1:::obj387:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d11b9b34:::obj309:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d11b9b34:::obj309:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d11b9b34:::obj309:head 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d11b9b34:::obj309:head v 107'309 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.422+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d0f049f1:::obj387:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d0f049f1:::obj387:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 113/113 lb 1:d0f049f1:::obj387:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d0f049f1:::obj387:head) v3 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0e799f1:::obj299:head local-lis/les=109/110 n=500 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d0f049f1:::obj387:head) 2026-02-11T23:31:17.422+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d0f049f1:::obj387:head) v3 finish latency 0.000018 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.422+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 119 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3e4a000 2026-02-11T23:31:17.422+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.422+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 120 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0b3f05c8:::obj589:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.422+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0b3f05c8:::obj589:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.422+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0b3f05c8:::obj589:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0b3f05c8:::obj589:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0bdade2e:::obj246:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0bdade2e:::obj246:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0bdade2e:::obj246:head 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0bdade2e:::obj246:head v 107'246 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.422+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0bdade2e:::obj246:head, version: 107'246, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bdade2e:::obj246:head@107'246, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0b3f05c8:::obj589:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0b3f05c8:::obj589:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 113/113 lb 4:0b3f05c8:::obj589:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0b3f05c8:::obj589:head) v3 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0a4fcbe7:::obj507:head local-lis/les=112/113 n=36 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0b3f05c8:::obj589:head) 2026-02-11T23:31:17.422+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0b3f05c8:::obj589:head) v3 finish latency 0.000014 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.422+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4911c80 con 0x3752000 2026-02-11T23:31:17.422+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d11b9b34:::obj309:head)]) -- 0x497e780 con 0x3752000 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.422+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4eab080 con 0x3e4a000 2026-02-11T23:31:17.422+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0bdade2e:::obj246:head)]) -- 0x4ba2480 con 0x3e4a000 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.422+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1043 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3752000 2026-02-11T23:31:17.422+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.422+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1044 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d11b9b34:::obj309:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3752000 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.422+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d11b9b34:::obj309:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.422+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d11b9b34:::obj309:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d11b9b34:::obj309:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d1337354:::obj10:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d1337354:::obj10:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d1337354:::obj10:head 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d1337354:::obj10:head v 107'10 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.422+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.422+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.422+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d11b9b34:::obj309:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d11b9b34:::obj309:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 113/113 lb 1:d11b9b34:::obj309:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d11b9b34:::obj309:head) v3 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d0f049f1:::obj387:head local-lis/les=109/110 n=501 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d11b9b34:::obj309:head) 2026-02-11T23:31:17.422+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.422+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d11b9b34:::obj309:head) v3 finish latency 0.000018 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.422+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.423+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 121 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3e4a000 2026-02-11T23:31:17.423+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.423+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 122 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0bdade2e:::obj246:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7700 con 0x3e4a000 2026-02-11T23:31:17.423+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0bdade2e:::obj246:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.423+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0bdade2e:::obj246:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0bdade2e:::obj246:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0bde2796:::obj413:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0bde2796:::obj413:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0bde2796:::obj413:head 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0bde2796:::obj413:head v 107'413 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.423+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0bde2796:::obj413:head, version: 107'413, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0bde2796:::obj413:head@107'413, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0bdade2e:::obj246:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0bdade2e:::obj246:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 113/113 lb 4:0bdade2e:::obj246:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0bdade2e:::obj246:head) v3 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0b3f05c8:::obj589:head local-lis/les=112/113 n=37 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0bdade2e:::obj246:head) 2026-02-11T23:31:17.423+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0bdade2e:::obj246:head) v3 finish latency 0.000015 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.423+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4911b00 con 0x3752000 2026-02-11T23:31:17.423+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d1337354:::obj10:head)]) -- 0x4954000 con 0x3752000 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.423+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4eaa780 con 0x3e4a000 2026-02-11T23:31:17.423+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0bde2796:::obj413:head)]) -- 0x4917b00 con 0x3e4a000 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.423+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1045 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43f2000 con 0x3752000 2026-02-11T23:31:17.423+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.423+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1046 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d1337354:::obj10:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5600 con 0x3752000 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.423+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d1337354:::obj10:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.423+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d1337354:::obj10:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d1337354:::obj10:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d1c8135f:::obj67:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d1c8135f:::obj67:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d1c8135f:::obj67:head 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d1c8135f:::obj67:head v 107'67 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.423+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d1337354:::obj10:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d1337354:::obj10:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 113/113 lb 1:d1337354:::obj10:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d1337354:::obj10:head) v3 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d11b9b34:::obj309:head local-lis/les=109/110 n=502 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d1337354:::obj10:head) 2026-02-11T23:31:17.423+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d1337354:::obj10:head) v3 finish latency 0.000019 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.423+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4911980 con 0x3752000 2026-02-11T23:31:17.423+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d1c8135f:::obj67:head)]) -- 0x4917500 con 0x3752000 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.423+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 123 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3e4a000 2026-02-11T23:31:17.423+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.423+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 124 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0bde2796:::obj413:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.423+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0bde2796:::obj413:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.423+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0bde2796:::obj413:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0bde2796:::obj413:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 113/112 [PushOp(4:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0c03281d:::obj596:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0c03281d:::obj596:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0c03281d:::obj596:head 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0c03281d:::obj596:head v 107'596 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.423+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0c03281d:::obj596:head, version: 107'596, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0c03281d:::obj596:head@107'596, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0bde2796:::obj413:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0bde2796:::obj413:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 113/113 lb 4:0bde2796:::obj413:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0bde2796:::obj413:head) v3 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bdade2e:::obj246:head local-lis/les=112/113 n=38 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0bde2796:::obj413:head) 2026-02-11T23:31:17.423+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.423+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0bde2796:::obj413:head) v3 finish latency 0.000017 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.423+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1047 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3752000 2026-02-11T23:31:17.423+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.423+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1048 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d1c8135f:::obj67:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f4000 con 0x3752000 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.423+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d1c8135f:::obj67:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.423+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d1c8135f:::obj67:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 113/109 [PushOp(1:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.423+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d1c8135f:::obj67:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d33f403b:::obj434:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.423+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d33f403b:::obj434:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.423+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d33f403b:::obj434:head 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d33f403b:::obj434:head v 107'434 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.424+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000094 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d1c8135f:::obj67:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d1c8135f:::obj67:head) v3 prio 15 cost 0 latency 0.000113 pg_backfill(progress 1.0 e 113/113 lb 1:d1c8135f:::obj67:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d1c8135f:::obj67:head) v3 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1337354:::obj10:head local-lis/les=109/110 n=503 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d1c8135f:::obj67:head) 2026-02-11T23:31:17.424+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d1c8135f:::obj67:head) v3 finish latency 0.000013 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.424+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4d67500 con 0x3e4a000 2026-02-11T23:31:17.424+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0c03281d:::obj596:head)]) -- 0x4954300 con 0x3e4a000 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.424+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4911800 con 0x3752000 2026-02-11T23:31:17.424+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d33f403b:::obj434:head)]) -- 0x4916300 con 0x3752000 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.424+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1049 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3752000 2026-02-11T23:31:17.424+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.424+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.424+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1050 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d33f403b:::obj434:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5080 con 0x3752000 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.424+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d33f403b:::obj434:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.424+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d33f403b:::obj434:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d33f403b:::obj434:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.424+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 125 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.424+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 113/109 [PushOp(1:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.424+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.424+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 126 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0c03281d:::obj596:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5bb80 con 0x3e4a000 2026-02-11T23:31:17.424+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0c03281d:::obj596:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.424+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0c03281d:::obj596:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d3f6948e:::obj218:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d3f6948e:::obj218:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d3f6948e:::obj218:head 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d3f6948e:::obj218:head v 107'218 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.424+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000078 MOSDPGPush(4.0 113/112 [PushOp(4:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d33f403b:::obj434:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d33f403b:::obj434:head) v3 prio 15 cost 0 latency 0.000115 pg_backfill(progress 1.0 e 113/113 lb 1:d33f403b:::obj434:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d33f403b:::obj434:head) v3 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d1c8135f:::obj67:head local-lis/les=109/110 n=504 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d33f403b:::obj434:head) 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0cfeb9e4:::obj357:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.424+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0cfeb9e4:::obj357:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0cfeb9e4:::obj357:head 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0cfeb9e4:::obj357:head v 107'357 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.424+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d33f403b:::obj434:head) v3 finish latency 0.000022 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0c03281d:::obj596:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.424+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0cfeb9e4:::obj357:head, version: 107'357, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0cfeb9e4:::obj357:head@107'357, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0c03281d:::obj596:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0c03281d:::obj596:head) v3 prio 15 cost 0 latency 0.000134 pg_backfill(progress 4.0 e 113/113 lb 4:0c03281d:::obj596:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0c03281d:::obj596:head) v3 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0bde2796:::obj413:head local-lis/les=112/113 n=39 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0c03281d:::obj596:head) 2026-02-11T23:31:17.424+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0c03281d:::obj596:head) v3 finish latency 0.000014 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.424+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4d66f00 con 0x3752000 2026-02-11T23:31:17.424+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d3f6948e:::obj218:head)]) -- 0x4916000 con 0x3752000 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.424+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.424+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.424+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4911680 con 0x3e4a000 2026-02-11T23:31:17.424+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0cfeb9e4:::obj357:head)]) -- 0x4916780 con 0x3e4a000 2026-02-11T23:31:17.424+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.425+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1051 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3752000 2026-02-11T23:31:17.425+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.425+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1052 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d3f6948e:::obj218:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3752000 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.425+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d3f6948e:::obj218:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.425+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d3f6948e:::obj218:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 113/109 [PushOp(1:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d3f6948e:::obj218:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d4006106:::obj329:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d4006106:::obj329:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d4006106:::obj329:head 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d4006106:::obj329:head v 107'329 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.425+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d3f6948e:::obj218:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d3f6948e:::obj218:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 113/113 lb 1:d3f6948e:::obj218:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d3f6948e:::obj218:head) v3 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d33f403b:::obj434:head local-lis/les=109/110 n=505 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d3f6948e:::obj218:head) 2026-02-11T23:31:17.425+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d3f6948e:::obj218:head) v3 finish latency 0.000017 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.425+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4d66c00 con 0x3752000 2026-02-11T23:31:17.425+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d4006106:::obj329:head)]) -- 0x4916480 con 0x3752000 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.425+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 127 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3e4a000 2026-02-11T23:31:17.425+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.425+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 128 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0cfeb9e4:::obj357:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3e4a000 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.425+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0cfeb9e4:::obj357:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.425+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0cfeb9e4:::obj357:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0cfeb9e4:::obj357:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000044 MOSDPGPush(4.0 113/112 [PushOp(4:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0dc4a75f:::obj282:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0dc4a75f:::obj282:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0dc4a75f:::obj282:head 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0dc4a75f:::obj282:head v 107'282 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.425+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0dc4a75f:::obj282:head, version: 107'282, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0dc4a75f:::obj282:head@107'282, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0cfeb9e4:::obj357:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0cfeb9e4:::obj357:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 113/113 lb 4:0cfeb9e4:::obj357:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0cfeb9e4:::obj357:head) v3 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0c03281d:::obj596:head local-lis/les=112/113 n=40 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0cfeb9e4:::obj357:head) 2026-02-11T23:31:17.425+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0cfeb9e4:::obj357:head) v3 finish latency 0.000015 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.425+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4911500 con 0x3e4a000 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.425+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0dc4a75f:::obj282:head)]) -- 0x4916c00 con 0x3e4a000 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.425+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1053 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea1a0 con 0x3752000 2026-02-11T23:31:17.425+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.425+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1054 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d4006106:::obj329:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3752000 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.425+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d4006106:::obj329:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.425+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d4006106:::obj329:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d4006106:::obj329:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d43593c2:::obj365:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d43593c2:::obj365:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d43593c2:::obj365:head 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d43593c2:::obj365:head v 107'365 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.425+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.425+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.425+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d4006106:::obj329:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d4006106:::obj329:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 113/113 lb 1:d4006106:::obj329:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d4006106:::obj329:head) v3 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d3f6948e:::obj218:head local-lis/les=109/110 n=506 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d4006106:::obj329:head) 2026-02-11T23:31:17.425+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.425+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d4006106:::obj329:head) v3 finish latency 0.000015 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.425+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.426+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4d66600 con 0x3752000 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 129 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3e4a000 2026-02-11T23:31:17.426+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d43593c2:::obj365:head)]) -- 0x4916900 con 0x3752000 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.426+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 130 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0dc4a75f:::obj282:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3e4a000 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.426+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0dc4a75f:::obj282:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.426+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0dc4a75f:::obj282:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 113/112 [PushOp(4:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0dc4a75f:::obj282:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0e39285d:::obj525:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0e39285d:::obj525:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0e39285d:::obj525:head 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0e39285d:::obj525:head v 107'525 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.426+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e39285d:::obj525:head, version: 107'525, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e39285d:::obj525:head@107'525, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0dc4a75f:::obj282:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0dc4a75f:::obj282:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 113/113 lb 4:0dc4a75f:::obj282:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0dc4a75f:::obj282:head) v3 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0cfeb9e4:::obj357:head local-lis/les=112/113 n=41 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0dc4a75f:::obj282:head) 2026-02-11T23:31:17.426+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0dc4a75f:::obj282:head) v3 finish latency 0.000017 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.426+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4911380 con 0x3e4a000 2026-02-11T23:31:17.426+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0e39285d:::obj525:head)]) -- 0x4916d80 con 0x3e4a000 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1055 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dd40 con 0x3752000 2026-02-11T23:31:17.426+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.426+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1056 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d43593c2:::obj365:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3752000 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.426+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d43593c2:::obj365:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.426+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d43593c2:::obj365:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d43593c2:::obj365:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d4c3bba7:::obj106:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d4c3bba7:::obj106:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d4c3bba7:::obj106:head 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d4c3bba7:::obj106:head v 107'106 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.426+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d43593c2:::obj365:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d43593c2:::obj365:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 113/113 lb 1:d43593c2:::obj365:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d43593c2:::obj365:head) v3 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4006106:::obj329:head local-lis/les=109/110 n=507 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d43593c2:::obj365:head) 2026-02-11T23:31:17.426+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d43593c2:::obj365:head) v3 finish latency 0.000017 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4d67b00 con 0x3752000 2026-02-11T23:31:17.426+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d4c3bba7:::obj106:head)]) -- 0x4954480 con 0x3752000 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 131 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516d860 con 0x3e4a000 2026-02-11T23:31:17.426+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.426+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 132 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0e39285d:::obj525:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3e4a000 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.426+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e39285d:::obj525:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.426+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e39285d:::obj525:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 113/112 [PushOp(4:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e39285d:::obj525:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0e3dc695:::obj78:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0e3dc695:::obj78:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0e3dc695:::obj78:head 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0e3dc695:::obj78:head v 107'78 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.426+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.426+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e3dc695:::obj78:head, version: 107'78, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e3dc695:::obj78:head@107'78, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e39285d:::obj525:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e39285d:::obj525:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 113/113 lb 4:0e39285d:::obj525:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0e39285d:::obj525:head) v3 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0dc4a75f:::obj282:head local-lis/les=112/113 n=42 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0e39285d:::obj525:head) 2026-02-11T23:31:17.426+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e39285d:::obj525:head) v3 finish latency 0.000017 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.426+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1057 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3752000 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.426+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4911200 con 0x3e4a000 2026-02-11T23:31:17.426+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.426+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0e3dc695:::obj78:head)]) -- 0x4916f00 con 0x3e4a000 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1058 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d4c3bba7:::obj106:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3752000 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.426+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d4c3bba7:::obj106:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d4c3bba7:::obj106:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 113/109 [PushOp(1:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.426+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d4c3bba7:::obj106:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.426+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d5f69272:::obj407:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.426+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d5f69272:::obj407:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d5f69272:::obj407:head 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d5f69272:::obj407:head v 107'407 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.427+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d4c3bba7:::obj106:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d4c3bba7:::obj106:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 1.0 e 113/113 lb 1:d4c3bba7:::obj106:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d4c3bba7:::obj106:head) v3 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d43593c2:::obj365:head local-lis/les=109/110 n=508 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d4c3bba7:::obj106:head) 2026-02-11T23:31:17.427+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d4c3bba7:::obj106:head) v3 finish latency 0.000016 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.427+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4d67800 con 0x3752000 2026-02-11T23:31:17.427+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d5f69272:::obj407:head)]) -- 0x4d66180 con 0x3752000 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.427+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 133 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cd00 con 0x3e4a000 2026-02-11T23:31:17.427+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.427+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.427+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 134 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0e3dc695:::obj78:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3e4a000 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.427+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e3dc695:::obj78:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.427+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e3dc695:::obj78:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 113/112 [PushOp(4:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e3dc695:::obj78:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0e5b0695:::obj117:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0e5b0695:::obj117:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0e5b0695:::obj117:head 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0e5b0695:::obj117:head v 107'117 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.427+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.427+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1059 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3752000 2026-02-11T23:31:17.427+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e5b0695:::obj117:head, version: 107'117, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5b0695:::obj117:head@107'117, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.427+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e3dc695:::obj78:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e3dc695:::obj78:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 113/113 lb 4:0e3dc695:::obj78:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.427+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1060 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d5f69272:::obj407:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3752000 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.427+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d5f69272:::obj407:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.427+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d5f69272:::obj407:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0e3dc695:::obj78:head) v3 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e39285d:::obj525:head local-lis/les=112/113 n=43 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0e3dc695:::obj78:head) 2026-02-11T23:31:17.427+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e3dc695:::obj78:head) v3 finish latency 0.000022 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d5f69272:::obj407:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d6df9426:::obj469:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d6df9426:::obj469:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d6df9426:::obj469:head 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d6df9426:::obj469:head v 107'469 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.427+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.427+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d5f69272:::obj407:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d5f69272:::obj407:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 113/113 lb 1:d5f69272:::obj407:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d5f69272:::obj407:head) v3 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d4c3bba7:::obj106:head local-lis/les=109/110 n=509 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d5f69272:::obj407:head) 2026-02-11T23:31:17.427+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d5f69272:::obj407:head) v3 finish latency 0.000015 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.427+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4910f00 con 0x3e4a000 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.427+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0e5b0695:::obj117:head)]) -- 0x49a2180 con 0x3e4a000 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.427+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.427+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.427+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c23980 con 0x3752000 2026-02-11T23:31:17.427+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d6df9426:::obj469:head)]) -- 0x4f2ca80 con 0x3752000 2026-02-11T23:31:17.427+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.428+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 135 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d6c0 con 0x3e4a000 2026-02-11T23:31:17.428+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.428+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 136 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0e5b0695:::obj117:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3e4a000 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.428+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e5b0695:::obj117:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.428+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e5b0695:::obj117:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 113/112 [PushOp(4:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e5b0695:::obj117:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0e5e5c87:::obj521:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0e5e5c87:::obj521:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0e5e5c87:::obj521:head 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0e5e5c87:::obj521:head v 107'521 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.428+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0e5e5c87:::obj521:head, version: 107'521, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0e5e5c87:::obj521:head@107'521, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e5b0695:::obj117:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e5b0695:::obj117:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 113/113 lb 4:0e5b0695:::obj117:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0e5b0695:::obj117:head) v3 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e3dc695:::obj78:head local-lis/les=112/113 n=44 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0e5b0695:::obj117:head) 2026-02-11T23:31:17.428+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e5b0695:::obj117:head) v3 finish latency 0.000017 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.428+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4910d80 con 0x3e4a000 2026-02-11T23:31:17.428+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0e5e5c87:::obj521:head)]) -- 0x4f2c900 con 0x3e4a000 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.428+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1061 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3752000 2026-02-11T23:31:17.428+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.428+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1062 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d6df9426:::obj469:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded080 con 0x3752000 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.428+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d6df9426:::obj469:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.428+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d6df9426:::obj469:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 113/109 [PushOp(1:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d6df9426:::obj469:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d80bb902:::obj418:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d80bb902:::obj418:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d80bb902:::obj418:head 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d80bb902:::obj418:head v 107'418 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.428+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d6df9426:::obj469:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d6df9426:::obj469:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 113/113 lb 1:d6df9426:::obj469:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d6df9426:::obj469:head) v3 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d5f69272:::obj407:head local-lis/les=109/110 n=510 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d6df9426:::obj469:head) 2026-02-11T23:31:17.428+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d6df9426:::obj469:head) v3 finish latency 0.000014 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.428+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 137 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3e4a000 2026-02-11T23:31:17.428+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.428+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 138 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0e5e5c87:::obj521:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3e4a000 2026-02-11T23:31:17.428+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e5e5c87:::obj521:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.428+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e5e5c87:::obj521:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e5e5c87:::obj521:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0ed26813:::obj311:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0ed26813:::obj311:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0ed26813:::obj311:head 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0ed26813:::obj311:head v 107'311 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.428+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.428+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0ed26813:::obj311:head, version: 107'311, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed26813:::obj311:head@107'311, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0e5e5c87:::obj521:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e5e5c87:::obj521:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 113/113 lb 4:0e5e5c87:::obj521:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0e5e5c87:::obj521:head) v3 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5b0695:::obj117:head local-lis/les=112/113 n=45 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0e5e5c87:::obj521:head) 2026-02-11T23:31:17.428+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0e5e5c87:::obj521:head) v3 finish latency 0.000015 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.428+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.428+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.428+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4ba2300 con 0x3752000 2026-02-11T23:31:17.428+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d80bb902:::obj418:head)]) -- 0x4f2c180 con 0x3752000 2026-02-11T23:31:17.428+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4910c00 con 0x3e4a000 2026-02-11T23:31:17.429+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0ed26813:::obj311:head)]) -- 0x4916780 con 0x3e4a000 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1063 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ebba0 con 0x3752000 2026-02-11T23:31:17.429+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.429+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.429+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1064 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d80bb902:::obj418:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3752000 2026-02-11T23:31:17.429+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d80bb902:::obj418:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d80bb902:::obj418:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 113/109 [PushOp(1:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d80bb902:::obj418:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d83b1cbf:::obj272:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d83b1cbf:::obj272:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d83b1cbf:::obj272:head 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d83b1cbf:::obj272:head v 107'272 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.429+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d80bb902:::obj418:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d80bb902:::obj418:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 113/113 lb 1:d80bb902:::obj418:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d80bb902:::obj418:head) v3 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d6df9426:::obj469:head local-lis/les=109/110 n=511 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d80bb902:::obj418:head) 2026-02-11T23:31:17.429+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d80bb902:::obj418:head) v3 finish latency 0.000013 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.429+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4910780 con 0x3752000 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d83b1cbf:::obj272:head)]) -- 0x4916000 con 0x3752000 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 139 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3e4a000 2026-02-11T23:31:17.429+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.429+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 140 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0ed26813:::obj311:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3e4a000 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.429+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0ed26813:::obj311:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.429+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0ed26813:::obj311:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 113/112 [PushOp(4:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0ed26813:::obj311:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0ed3c684:::obj270:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0ed3c684:::obj270:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0ed3c684:::obj270:head 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0ed3c684:::obj270:head v 107'270 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.429+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0ed3c684:::obj270:head, version: 107'270, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ed3c684:::obj270:head@107'270, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0ed26813:::obj311:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0ed26813:::obj311:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 113/113 lb 4:0ed26813:::obj311:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0ed26813:::obj311:head) v3 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0e5e5c87:::obj521:head local-lis/les=112/113 n=46 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0ed26813:::obj311:head) 2026-02-11T23:31:17.429+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0ed26813:::obj311:head) v3 finish latency 0.000016 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4910a80 con 0x3e4a000 2026-02-11T23:31:17.429+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0ed3c684:::obj270:head)]) -- 0x4916300 con 0x3e4a000 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1065 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3752000 2026-02-11T23:31:17.429+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.429+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1066 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d83b1cbf:::obj272:head) ==== 1062+0+0 (unknown 0 0 0) 0x4deec00 con 0x3752000 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.429+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d83b1cbf:::obj272:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.429+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d83b1cbf:::obj272:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d83b1cbf:::obj272:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d85a09a6:::obj441:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d85a09a6:::obj441:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d85a09a6:::obj441:head 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d85a09a6:::obj441:head v 107'441 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.429+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.429+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.429+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d83b1cbf:::obj272:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d83b1cbf:::obj272:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 113/113 lb 1:d83b1cbf:::obj272:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d83b1cbf:::obj272:head) v3 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d80bb902:::obj418:head local-lis/les=109/110 n=512 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d83b1cbf:::obj272:head) 2026-02-11T23:31:17.429+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.429+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d83b1cbf:::obj272:head) v3 finish latency 0.000016 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.429+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.430+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4e71080 con 0x3752000 2026-02-11T23:31:17.430+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d85a09a6:::obj441:head)]) -- 0x4954300 con 0x3752000 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.430+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 141 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3e4a000 2026-02-11T23:31:17.430+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.430+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.430+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 142 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0ed3c684:::obj270:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def180 con 0x3e4a000 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.430+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0ed3c684:::obj270:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.430+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0ed3c684:::obj270:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 113/112 [PushOp(4:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0ed3c684:::obj270:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0ee9ae15:::obj4:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0ee9ae15:::obj4:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0ee9ae15:::obj4:head 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0ee9ae15:::obj4:head v 107'4 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.430+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0ee9ae15:::obj4:head, version: 107'4, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0ee9ae15:::obj4:head@107'4, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0ed3c684:::obj270:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0ed3c684:::obj270:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 113/113 lb 4:0ed3c684:::obj270:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0ed3c684:::obj270:head) v3 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed26813:::obj311:head local-lis/les=112/113 n=47 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0ed3c684:::obj270:head) 2026-02-11T23:31:17.430+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0ed3c684:::obj270:head) v3 finish latency 0.000014 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.430+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1067 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3752000 2026-02-11T23:31:17.430+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.430+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.430+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1068 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d85a09a6:::obj441:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3752000 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.430+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d85a09a6:::obj441:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.430+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d85a09a6:::obj441:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 113/109 [PushOp(1:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d85a09a6:::obj441:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d881fc7d:::obj242:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d881fc7d:::obj242:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d881fc7d:::obj242:head 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d881fc7d:::obj242:head v 107'242 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.430+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.430+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d85a09a6:::obj441:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d85a09a6:::obj441:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 113/113 lb 1:d85a09a6:::obj441:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d85a09a6:::obj441:head) v3 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d83b1cbf:::obj272:head local-lis/les=109/110 n=513 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d85a09a6:::obj441:head) 2026-02-11T23:31:17.430+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d85a09a6:::obj441:head) v3 finish latency 0.000013 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.430+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4a04000 con 0x3e4a000 2026-02-11T23:31:17.430+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0ee9ae15:::obj4:head)]) -- 0x4917500 con 0x3e4a000 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.430+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.430+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4a04300 con 0x3752000 2026-02-11T23:31:17.430+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.430+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d881fc7d:::obj242:head)]) -- 0x4911980 con 0x3752000 2026-02-11T23:31:17.430+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.431+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 143 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2340 con 0x3e4a000 2026-02-11T23:31:17.431+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 113 2026-02-11T23:31:17.431+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 144 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0ee9ae15:::obj4:head) ==== 1060+0+0 (unknown 0 0 0) 0x5164c00 con 0x3e4a000 2026-02-11T23:31:17.431+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0ee9ae15:::obj4:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.431+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0ee9ae15:::obj4:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0ee9ae15:::obj4:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 113/112 [PushOp(4:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0efd4012:::obj445:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0efd4012:::obj445:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0efd4012:::obj445:head 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0efd4012:::obj445:head v 107'445 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.431+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.431+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1069 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3752000 2026-02-11T23:31:17.431+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.431+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.431+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1070 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d881fc7d:::obj242:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165180 con 0x3752000 2026-02-11T23:31:17.431+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d881fc7d:::obj242:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.431+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d881fc7d:::obj242:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0efd4012:::obj445:head, version: 107'445, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0efd4012:::obj445:head@107'445, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 113/109 [PushOp(1:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0ee9ae15:::obj4:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0ee9ae15:::obj4:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 113/113 lb 4:0ee9ae15:::obj4:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0ee9ae15:::obj4:head) v3 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ed3c684:::obj270:head local-lis/les=112/113 n=48 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0ee9ae15:::obj4:head) 2026-02-11T23:31:17.431+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0ee9ae15:::obj4:head) v3 finish latency 0.000018 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d9239af3:::obj409:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d881fc7d:::obj242:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d9239af3:::obj409:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d9239af3:::obj409:head 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d9239af3:::obj409:head v 107'409 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.431+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d881fc7d:::obj242:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d881fc7d:::obj242:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 113/113 lb 1:d881fc7d:::obj242:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d881fc7d:::obj242:head) v3 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d85a09a6:::obj441:head local-lis/les=109/110 n=514 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d881fc7d:::obj242:head) 2026-02-11T23:31:17.431+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d881fc7d:::obj242:head) v3 finish latency 0.000023 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.431+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4a04180 con 0x3e4a000 2026-02-11T23:31:17.431+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0efd4012:::obj445:head)]) -- 0x4eaa780 con 0x3e4a000 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.431+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4a04480 con 0x3752000 2026-02-11T23:31:17.431+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d9239af3:::obj409:head)]) -- 0x4911b00 con 0x3752000 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.431+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 145 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.431+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.431+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 146 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0efd4012:::obj445:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3e4a000 2026-02-11T23:31:17.431+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0efd4012:::obj445:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.431+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0efd4012:::obj445:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0efd4012:::obj445:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0f33f807:::obj273:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0f33f807:::obj273:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0f33f807:::obj273:head 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0f33f807:::obj273:head v 107'273 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.431+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1071 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3752000 2026-02-11T23:31:17.431+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.431+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.431+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.431+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1072 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d9239af3:::obj409:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3752000 2026-02-11T23:31:17.431+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d9239af3:::obj409:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.431+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d9239af3:::obj409:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0f33f807:::obj273:head, version: 107'273, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f33f807:::obj273:head@107'273, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0efd4012:::obj445:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0efd4012:::obj445:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 113/113 lb 4:0efd4012:::obj445:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 113/109 [PushOp(1:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0efd4012:::obj445:head) v3 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0ee9ae15:::obj4:head local-lis/les=112/113 n=49 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0efd4012:::obj445:head) 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.431+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d99b7c45:::obj564:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d99b7c45:::obj564:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0efd4012:::obj445:head) v3 finish latency 0.000020 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d99b7c45:::obj564:head 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d99b7c45:::obj564:head v 107'564 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d9239af3:::obj409:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.431+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.431+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.431+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d9239af3:::obj409:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d9239af3:::obj409:head) v3 prio 15 cost 0 latency 0.000102 pg_backfill(progress 1.0 e 113/113 lb 1:d9239af3:::obj409:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d9239af3:::obj409:head) v3 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d881fc7d:::obj242:head local-lis/les=109/110 n=515 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d9239af3:::obj409:head) 2026-02-11T23:31:17.431+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d9239af3:::obj409:head) v3 finish latency 0.000015 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.431+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.431+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4a04600 con 0x3e4a000 2026-02-11T23:31:17.431+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0f33f807:::obj273:head)]) -- 0x4eab080 con 0x3e4a000 2026-02-11T23:31:17.431+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.432+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4a04780 con 0x3752000 2026-02-11T23:31:17.432+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d99b7c45:::obj564:head)]) -- 0x4911c80 con 0x3752000 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.432+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 147 ==== MOSDPGPush(4.0 113/112 [PushOp(4:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.432+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.432+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.432+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 148 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0f33f807:::obj273:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.432+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0f33f807:::obj273:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.432+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0f33f807:::obj273:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 113/112 [PushOp(4:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0f33f807:::obj273:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:0f45d4cb:::obj171:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:0f45d4cb:::obj171:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:0f45d4cb:::obj171:head 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:0f45d4cb:::obj171:head v 107'171 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.432+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.432+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1073 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3752000 2026-02-11T23:31:17.432+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.432+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:0f45d4cb:::obj171:head, version: 107'171, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:0f45d4cb:::obj171:head@107'171, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.432+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1074 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d99b7c45:::obj564:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3752000 2026-02-11T23:31:17.432+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d99b7c45:::obj564:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.432+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d99b7c45:::obj564:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0f33f807:::obj273:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 113/109 [PushOp(1:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0f33f807:::obj273:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 113/113 lb 4:0f33f807:::obj273:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0f33f807:::obj273:head) v3 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0efd4012:::obj445:head local-lis/les=112/113 n=50 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0f33f807:::obj273:head) 2026-02-11T23:31:17.432+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d9c98c7a:::obj262:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d9c98c7a:::obj262:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0f33f807:::obj273:head) v3 finish latency 0.000018 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d9c98c7a:::obj262:head 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d9c98c7a:::obj262:head v 107'262 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d99b7c45:::obj564:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.432+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.432+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000075 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d99b7c45:::obj564:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d99b7c45:::obj564:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 113/113 lb 1:d99b7c45:::obj564:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d99b7c45:::obj564:head) v3 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9239af3:::obj409:head local-lis/les=109/110 n=516 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d99b7c45:::obj564:head) 2026-02-11T23:31:17.432+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d99b7c45:::obj564:head) v3 finish latency 0.000015 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.432+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4a04c00 con 0x3e4a000 2026-02-11T23:31:17.432+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:0f45d4cb:::obj171:head)]) -- 0x4eabc80 con 0x3e4a000 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.432+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.432+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.432+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4a04900 con 0x3752000 2026-02-11T23:31:17.432+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d9c98c7a:::obj262:head)]) -- 0x4910180 con 0x3752000 2026-02-11T23:31:17.432+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.433+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 149 ==== MOSDPGPush(4.0 113/112 [PushOp(4:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x516cea0 con 0x3e4a000 2026-02-11T23:31:17.433+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.433+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.433+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 150 ==== pg_backfill(progress 4.0 e 113/113 lb 4:0f45d4cb:::obj171:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.433+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:0f45d4cb:::obj171:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.433+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0f45d4cb:::obj171:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.433+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1075 ==== MOSDPGPush(1.0 113/109 [PushOp(1:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182680 con 0x3752000 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0f45d4cb:::obj171:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.433+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000005 epoch 113 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.433+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.433+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1076 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d9c98c7a:::obj262:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd700 con 0x3752000 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.433+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d9c98c7a:::obj262:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.433+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d9c98c7a:::obj262:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:104778fc:::obj2:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:104778fc:::obj2:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:104778fc:::obj2:head 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:104778fc:::obj2:head v 107'2 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.433+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:104778fc:::obj2:head, version: 107'2, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:104778fc:::obj2:head@107'2, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000078 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000094 MOSDPGPush(1.0 113/109 [PushOp(1:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:0f45d4cb:::obj171:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0f45d4cb:::obj171:head) v3 prio 15 cost 0 latency 0.000115 pg_backfill(progress 4.0 e 113/113 lb 4:0f45d4cb:::obj171:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:0f45d4cb:::obj171:head) v3 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f33f807:::obj273:head local-lis/les=112/113 n=51 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:0f45d4cb:::obj171:head) 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.433+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:d9da6b63:::obj302:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:0f45d4cb:::obj171:head) v3 finish latency 0.000017 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:d9da6b63:::obj302:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:d9da6b63:::obj302:head 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d9c98c7a:::obj262:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:d9da6b63:::obj302:head v 107'302 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.433+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d9c98c7a:::obj262:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d9c98c7a:::obj262:head) v3 prio 15 cost 0 latency 0.000155 pg_backfill(progress 1.0 e 113/113 lb 1:d9c98c7a:::obj262:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d9c98c7a:::obj262:head) v3 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d99b7c45:::obj564:head local-lis/les=109/110 n=517 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d9c98c7a:::obj262:head) 2026-02-11T23:31:17.433+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d9c98c7a:::obj262:head) v3 finish latency 0.000016 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.433+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49b8780 con 0x3e4a000 2026-02-11T23:31:17.433+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:104778fc:::obj2:head)]) -- 0x4910000 con 0x3e4a000 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.433+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.433+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49b8900 con 0x3752000 2026-02-11T23:31:17.433+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:d9da6b63:::obj302:head)]) -- 0x497e180 con 0x3752000 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.433+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 151 ==== MOSDPGPush(4.0 113/112 [PushOp(4:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516da00 con 0x3e4a000 2026-02-11T23:31:17.433+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 113 2026-02-11T23:31:17.433+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.433+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 152 ==== pg_backfill(progress 4.0 e 113/113 lb 4:104778fc:::obj2:head) ==== 1060+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.433+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:104778fc:::obj2:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.433+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:104778fc:::obj2:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 113/112 [PushOp(4:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.433+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:104778fc:::obj2:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:105983a6:::obj88:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.433+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:105983a6:::obj88:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:105983a6:::obj88:head 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:105983a6:::obj88:head v 107'88 2026-02-11T23:31:17.433+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.433+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.433+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1077 ==== MOSDPGPush(1.0 113/109 [PushOp(1:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5182000 con 0x3752000 2026-02-11T23:31:17.433+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.433+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.433+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1078 ==== pg_backfill(progress 1.0 e 113/113 lb 1:d9da6b63:::obj302:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3752000 2026-02-11T23:31:17.433+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:d9da6b63:::obj302:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.433+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d9da6b63:::obj302:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:105983a6:::obj88:head, version: 107'88, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:105983a6:::obj88:head@107'88, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:104778fc:::obj2:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000049 MOSDPGPush(1.0 113/109 [PushOp(1:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:104778fc:::obj2:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 4.0 e 113/113 lb 4:104778fc:::obj2:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:104778fc:::obj2:head) v3 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:0f45d4cb:::obj171:head local-lis/les=112/113 n=52 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:104778fc:::obj2:head) 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.434+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:da0e2ac9:::obj69:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:da0e2ac9:::obj69:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:104778fc:::obj2:head) v3 finish latency 0.000018 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:da0e2ac9:::obj69:head 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d9da6b63:::obj302:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:da0e2ac9:::obj69:head v 107'69 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.434+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000076 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:d9da6b63:::obj302:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d9da6b63:::obj302:head) v3 prio 15 cost 0 latency 0.000117 pg_backfill(progress 1.0 e 113/113 lb 1:d9da6b63:::obj302:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:d9da6b63:::obj302:head) v3 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9c98c7a:::obj262:head local-lis/les=109/110 n=518 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:d9da6b63:::obj302:head) 2026-02-11T23:31:17.434+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:d9da6b63:::obj302:head) v3 finish latency 0.000015 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.434+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49b8c00 con 0x3e4a000 2026-02-11T23:31:17.434+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:105983a6:::obj88:head)]) -- 0x497e480 con 0x3e4a000 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.434+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49b8f00 con 0x3752000 2026-02-11T23:31:17.434+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:da0e2ac9:::obj69:head)]) -- 0x4910600 con 0x3752000 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 153 ==== MOSDPGPush(4.0 113/112 [PushOp(4:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51824e0 con 0x3e4a000 2026-02-11T23:31:17.434+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.434+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 154 ==== pg_backfill(progress 4.0 e 113/113 lb 4:105983a6:::obj88:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.434+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:105983a6:::obj88:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.434+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:105983a6:::obj88:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:105983a6:::obj88:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:10754df1:::obj159:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:10754df1:::obj159:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:10754df1:::obj159:head 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:10754df1:::obj159:head v 107'159 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.434+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:10754df1:::obj159:head, version: 107'159, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10754df1:::obj159:head@107'159, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:105983a6:::obj88:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:105983a6:::obj88:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 113/113 lb 4:105983a6:::obj88:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:105983a6:::obj88:head) v3 2026-02-11T23:31:17.434+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1079 ==== MOSDPGPush(1.0 113/109 [PushOp(1:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3752000 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:104778fc:::obj2:head local-lis/les=112/113 n=53 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:105983a6:::obj88:head) 2026-02-11T23:31:17.434+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.434+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.434+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:105983a6:::obj88:head) v3 finish latency 0.000017 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1080 ==== pg_backfill(progress 1.0 e 113/113 lb 1:da0e2ac9:::obj69:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164100 con 0x3752000 2026-02-11T23:31:17.434+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:da0e2ac9:::obj69:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.434+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:da0e2ac9:::obj69:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:da0e2ac9:::obj69:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:da920364:::obj185:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:da920364:::obj185:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:da920364:::obj185:head 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:da920364:::obj185:head v 107'185 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.434+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.434+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:da0e2ac9:::obj69:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:da0e2ac9:::obj69:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 113/113 lb 1:da0e2ac9:::obj69:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:da0e2ac9:::obj69:head) v3 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:d9da6b63:::obj302:head local-lis/les=109/110 n=519 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:da0e2ac9:::obj69:head) 2026-02-11T23:31:17.434+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:da0e2ac9:::obj69:head) v3 finish latency 0.000015 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.434+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49b9080 con 0x3e4a000 2026-02-11T23:31:17.434+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:10754df1:::obj159:head)]) -- 0x4917c80 con 0x3e4a000 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.434+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.434+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49b9380 con 0x3752000 2026-02-11T23:31:17.434+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:da920364:::obj185:head)]) -- 0x497e000 con 0x3752000 2026-02-11T23:31:17.434+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.435+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1081 ==== MOSDPGPush(1.0 113/109 [PushOp(1:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2680 con 0x3752000 2026-02-11T23:31:17.435+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.435+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.435+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1082 ==== pg_backfill(progress 1.0 e 113/113 lb 1:da920364:::obj185:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163b80 con 0x3752000 2026-02-11T23:31:17.435+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:da920364:::obj185:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.435+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:da920364:::obj185:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 113/109 [PushOp(1:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:da920364:::obj185:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:da932e1e:::obj333:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.435+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 155 ==== MOSDPGPush(4.0 113/112 [PushOp(4:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:da932e1e:::obj333:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.435+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:da932e1e:::obj333:head 2026-02-11T23:31:17.435+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:da932e1e:::obj333:head v 107'333 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.435+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.435+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 156 ==== pg_backfill(progress 4.0 e 113/113 lb 4:10754df1:::obj159:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163600 con 0x3e4a000 2026-02-11T23:31:17.435+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:10754df1:::obj159:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.435+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10754df1:::obj159:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:da920364:::obj185:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:da920364:::obj185:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 1.0 e 113/113 lb 1:da920364:::obj185:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000061 MOSDPGPush(4.0 113/112 [PushOp(4:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:da920364:::obj185:head) v3 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da0e2ac9:::obj69:head local-lis/les=109/110 n=520 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:da920364:::obj185:head) 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.435+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:10b422cc:::obj197:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:da920364:::obj185:head) v3 finish latency 0.000018 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:10b422cc:::obj197:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10754df1:::obj159:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:10b422cc:::obj197:head 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:10b422cc:::obj197:head v 107'197 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.435+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.435+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:10b422cc:::obj197:head, version: 107'197, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10b422cc:::obj197:head@107'197, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10754df1:::obj159:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:10754df1:::obj159:head) v3 prio 15 cost 0 latency 0.000114 pg_backfill(progress 4.0 e 113/113 lb 4:10754df1:::obj159:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:10754df1:::obj159:head) v3 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:105983a6:::obj88:head local-lis/les=112/113 n=54 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:10754df1:::obj159:head) 2026-02-11T23:31:17.435+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:10754df1:::obj159:head) v3 finish latency 0.000013 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.435+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49b9680 con 0x3752000 2026-02-11T23:31:17.435+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:da932e1e:::obj333:head)]) -- 0x4917500 con 0x3752000 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.435+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.435+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.435+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49d6180 con 0x3e4a000 2026-02-11T23:31:17.435+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:10b422cc:::obj197:head)]) -- 0x4954300 con 0x3e4a000 2026-02-11T23:31:17.435+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1083 ==== MOSDPGPush(1.0 113/109 [PushOp(1:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3752000 2026-02-11T23:31:17.436+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.436+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1084 ==== pg_backfill(progress 1.0 e 113/113 lb 1:da932e1e:::obj333:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163080 con 0x3752000 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.436+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:da932e1e:::obj333:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.436+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:da932e1e:::obj333:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:da932e1e:::obj333:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000046 MOSDPGPush(1.0 113/109 [PushOp(1:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:dbca42a3:::obj382:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:dbca42a3:::obj382:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:dbca42a3:::obj382:head 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:dbca42a3:::obj382:head v 107'382 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.436+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:da932e1e:::obj333:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:da932e1e:::obj333:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 1.0 e 113/113 lb 1:da932e1e:::obj333:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:da932e1e:::obj333:head) v3 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da920364:::obj185:head local-lis/les=109/110 n=521 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:da932e1e:::obj333:head) 2026-02-11T23:31:17.436+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:da932e1e:::obj333:head) v3 finish latency 0.000017 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 157 ==== MOSDPGPush(4.0 113/112 [PushOp(4:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.436+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.436+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d6600 con 0x3752000 2026-02-11T23:31:17.436+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.436+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:dbca42a3:::obj382:head)]) -- 0x4916300 con 0x3752000 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.436+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 158 ==== pg_backfill(progress 4.0 e 113/113 lb 4:10b422cc:::obj197:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3e4a000 2026-02-11T23:31:17.436+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:10b422cc:::obj197:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10b422cc:::obj197:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 113/112 [PushOp(4:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10b422cc:::obj197:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:10c2d3a1:::obj448:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:10c2d3a1:::obj448:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:10c2d3a1:::obj448:head 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:10c2d3a1:::obj448:head v 107'448 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.436+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:10c2d3a1:::obj448:head, version: 107'448, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10c2d3a1:::obj448:head@107'448, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10b422cc:::obj197:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:10b422cc:::obj197:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 113/113 lb 4:10b422cc:::obj197:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:10b422cc:::obj197:head) v3 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10754df1:::obj159:head local-lis/les=112/113 n=55 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:10b422cc:::obj197:head) 2026-02-11T23:31:17.436+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:10b422cc:::obj197:head) v3 finish latency 0.000024 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.436+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49d6780 con 0x3e4a000 2026-02-11T23:31:17.436+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:10c2d3a1:::obj448:head)]) -- 0x49d6300 con 0x3e4a000 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1085 ==== MOSDPGPush(1.0 113/109 [PushOp(1:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3752000 2026-02-11T23:31:17.436+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.436+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1086 ==== pg_backfill(progress 1.0 e 113/113 lb 1:dbca42a3:::obj382:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162580 con 0x3752000 2026-02-11T23:31:17.436+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:dbca42a3:::obj382:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.436+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dbca42a3:::obj382:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dbca42a3:::obj382:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 113/109 [PushOp(1:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:dc49c339:::obj378:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:dc49c339:::obj378:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:dc49c339:::obj378:head 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:dc49c339:::obj378:head v 107'378 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.436+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.436+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dbca42a3:::obj382:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dbca42a3:::obj382:head) v3 prio 15 cost 0 latency 0.000103 pg_backfill(progress 1.0 e 113/113 lb 1:dbca42a3:::obj382:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:dbca42a3:::obj382:head) v3 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:da932e1e:::obj333:head local-lis/les=109/110 n=522 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:dbca42a3:::obj382:head) 2026-02-11T23:31:17.436+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dbca42a3:::obj382:head) v3 finish latency 0.000017 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.436+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d6f00 con 0x3752000 2026-02-11T23:31:17.436+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:dc49c339:::obj378:head)]) -- 0x4916000 con 0x3752000 2026-02-11T23:31:17.436+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.436+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 159 ==== MOSDPGPush(4.0 113/112 [PushOp(4:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2340 con 0x3e4a000 2026-02-11T23:31:17.436+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.436+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.437+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 160 ==== pg_backfill(progress 4.0 e 113/113 lb 4:10c2d3a1:::obj448:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162000 con 0x3e4a000 2026-02-11T23:31:17.437+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:10c2d3a1:::obj448:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10c2d3a1:::obj448:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10c2d3a1:::obj448:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:10f4e680:::obj204:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:10f4e680:::obj204:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:10f4e680:::obj204:head 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:10f4e680:::obj204:head v 107'204 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.437+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:10f4e680:::obj204:head, version: 107'204, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:10f4e680:::obj204:head@107'204, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10c2d3a1:::obj448:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:10c2d3a1:::obj448:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 113/113 lb 4:10c2d3a1:::obj448:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:10c2d3a1:::obj448:head) v3 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10b422cc:::obj197:head local-lis/les=112/113 n=56 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:10c2d3a1:::obj448:head) 2026-02-11T23:31:17.437+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:10c2d3a1:::obj448:head) v3 finish latency 0.000023 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1087 ==== MOSDPGPush(1.0 113/109 [PushOp(1:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3752000 2026-02-11T23:31:17.437+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.437+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1088 ==== pg_backfill(progress 1.0 e 113/113 lb 1:dc49c339:::obj378:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3752000 2026-02-11T23:31:17.437+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc49c339:::obj378:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.437+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc49c339:::obj378:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc49c339:::obj378:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:dc6ed3c7:::obj113:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:dc6ed3c7:::obj113:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:dc6ed3c7:::obj113:head 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:dc6ed3c7:::obj113:head v 107'113 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.437+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc49c339:::obj378:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc49c339:::obj378:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 113/113 lb 1:dc49c339:::obj378:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:dc49c339:::obj378:head) v3 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dbca42a3:::obj382:head local-lis/les=109/110 n=523 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:dc49c339:::obj378:head) 2026-02-11T23:31:17.437+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc49c339:::obj378:head) v3 finish latency 0.000015 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.437+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49d6c00 con 0x3e4a000 2026-02-11T23:31:17.437+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:10f4e680:::obj204:head)]) -- 0x4916780 con 0x3e4a000 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d7080 con 0x3752000 2026-02-11T23:31:17.437+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:dc6ed3c7:::obj113:head)]) -- 0x4f2c180 con 0x3752000 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 161 ==== MOSDPGPush(4.0 113/112 [PushOp(4:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3e4a000 2026-02-11T23:31:17.437+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.437+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 162 ==== pg_backfill(progress 4.0 e 113/113 lb 4:10f4e680:::obj204:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3e4a000 2026-02-11T23:31:17.437+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:10f4e680:::obj204:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.437+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10f4e680:::obj204:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 113/112 [PushOp(4:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10f4e680:::obj204:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:11426d25:::obj314:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:11426d25:::obj314:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:11426d25:::obj314:head 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:11426d25:::obj314:head v 107'314 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.437+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:11426d25:::obj314:head, version: 107'314, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11426d25:::obj314:head@107'314, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:10f4e680:::obj204:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:10f4e680:::obj204:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 113/113 lb 4:10f4e680:::obj204:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:10f4e680:::obj204:head) v3 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10c2d3a1:::obj448:head local-lis/les=112/113 n=57 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:10f4e680:::obj204:head) 2026-02-11T23:31:17.437+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:10f4e680:::obj204:head) v3 finish latency 0.000018 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1089 ==== MOSDPGPush(1.0 113/109 [PushOp(1:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3752000 2026-02-11T23:31:17.437+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.437+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1090 ==== pg_backfill(progress 1.0 e 113/113 lb 1:dc6ed3c7:::obj113:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3752000 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.437+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc6ed3c7:::obj113:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.437+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc6ed3c7:::obj113:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc6ed3c7:::obj113:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:dc6fad89:::obj337:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:dc6fad89:::obj337:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:dc6fad89:::obj337:head 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:dc6fad89:::obj337:head v 107'337 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.437+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.437+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.437+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc6ed3c7:::obj113:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc6ed3c7:::obj113:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 113/113 lb 1:dc6ed3c7:::obj113:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:dc6ed3c7:::obj113:head) v3 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc49c339:::obj378:head local-lis/les=109/110 n=524 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:dc6ed3c7:::obj113:head) 2026-02-11T23:31:17.437+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc6ed3c7:::obj113:head) v3 finish latency 0.000015 2026-02-11T23:31:17.437+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.437+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49d7380 con 0x3e4a000 2026-02-11T23:31:17.437+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:11426d25:::obj314:head)]) -- 0x4f2c900 con 0x3e4a000 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.437+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.438+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d7200 con 0x3752000 2026-02-11T23:31:17.438+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:dc6fad89:::obj337:head)]) -- 0x4f2ca80 con 0x3752000 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.438+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 163 ==== MOSDPGPush(4.0 113/112 [PushOp(4:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3e4a000 2026-02-11T23:31:17.438+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.438+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.438+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 164 ==== pg_backfill(progress 4.0 e 113/113 lb 4:11426d25:::obj314:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3e4a000 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.438+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:11426d25:::obj314:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.438+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:11426d25:::obj314:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 113/112 [PushOp(4:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:11426d25:::obj314:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:11a85aa0:::obj43:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:11a85aa0:::obj43:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:11a85aa0:::obj43:head 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:11a85aa0:::obj43:head v 107'43 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.438+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:11a85aa0:::obj43:head, version: 107'43, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11a85aa0:::obj43:head@107'43, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:11426d25:::obj314:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:11426d25:::obj314:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 113/113 lb 4:11426d25:::obj314:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:11426d25:::obj314:head) v3 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:10f4e680:::obj204:head local-lis/les=112/113 n=58 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:11426d25:::obj314:head) 2026-02-11T23:31:17.438+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:11426d25:::obj314:head) v3 finish latency 0.000019 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.438+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1091 ==== MOSDPGPush(1.0 113/109 [PushOp(1:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ebba0 con 0x3752000 2026-02-11T23:31:17.438+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.438+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.438+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1092 ==== pg_backfill(progress 1.0 e 113/113 lb 1:dc6fad89:::obj337:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d700 con 0x3752000 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.438+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc6fad89:::obj337:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.438+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc6fad89:::obj337:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 113/109 [PushOp(1:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc6fad89:::obj337:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:dc7d10b2:::obj533:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:dc7d10b2:::obj533:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:dc7d10b2:::obj533:head 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:dc7d10b2:::obj533:head v 107'533 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.438+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.438+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc6fad89:::obj337:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc6fad89:::obj337:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 113/113 lb 1:dc6fad89:::obj337:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:dc6fad89:::obj337:head) v3 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6ed3c7:::obj113:head local-lis/les=109/110 n=525 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:dc6fad89:::obj337:head) 2026-02-11T23:31:17.438+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc6fad89:::obj337:head) v3 finish latency 0.000017 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.438+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49d7800 con 0x3e4a000 2026-02-11T23:31:17.438+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:11a85aa0:::obj43:head)]) -- 0x49a2180 con 0x3e4a000 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.438+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.438+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.438+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d7500 con 0x3752000 2026-02-11T23:31:17.438+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:dc7d10b2:::obj533:head)]) -- 0x4910f00 con 0x3752000 2026-02-11T23:31:17.438+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.439+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1093 ==== MOSDPGPush(1.0 113/109 [PushOp(1:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3752000 2026-02-11T23:31:17.439+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.439+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1094 ==== pg_backfill(progress 1.0 e 113/113 lb 1:dc7d10b2:::obj533:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3752000 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.439+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc7d10b2:::obj533:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.439+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc7d10b2:::obj533:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 113/109 [PushOp(1:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc7d10b2:::obj533:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:dc9e4e8c:::obj567:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.439+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 165 ==== MOSDPGPush(4.0 113/112 [PushOp(4:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:dc9e4e8c:::obj567:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:dc9e4e8c:::obj567:head 2026-02-11T23:31:17.439+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:dc9e4e8c:::obj567:head v 107'567 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.439+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.439+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.439+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 166 ==== pg_backfill(progress 4.0 e 113/113 lb 4:11a85aa0:::obj43:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3e4a000 2026-02-11T23:31:17.439+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:11a85aa0:::obj43:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.439+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:11a85aa0:::obj43:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc7d10b2:::obj533:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000055 MOSDPGPush(4.0 113/112 [PushOp(4:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc7d10b2:::obj533:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 113/113 lb 1:dc7d10b2:::obj533:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:dc7d10b2:::obj533:head) v3 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc6fad89:::obj337:head local-lis/les=109/110 n=526 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:dc7d10b2:::obj533:head) 2026-02-11T23:31:17.439+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc7d10b2:::obj533:head) v3 finish latency 0.000020 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:11bee109:::obj319:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:11a85aa0:::obj43:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:11bee109:::obj319:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:11bee109:::obj319:head 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:11bee109:::obj319:head v 107'319 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.439+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:11bee109:::obj319:head, version: 107'319, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:11bee109:::obj319:head@107'319, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:11a85aa0:::obj43:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:11a85aa0:::obj43:head) v3 prio 15 cost 0 latency 0.000121 pg_backfill(progress 4.0 e 113/113 lb 4:11a85aa0:::obj43:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:11a85aa0:::obj43:head) v3 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11426d25:::obj314:head local-lis/les=112/113 n=59 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:11a85aa0:::obj43:head) 2026-02-11T23:31:17.439+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:11a85aa0:::obj43:head) v3 finish latency 0.000014 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.439+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d7980 con 0x3752000 2026-02-11T23:31:17.439+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:dc9e4e8c:::obj567:head)]) -- 0x4d67800 con 0x3752000 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.439+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49d7e00 con 0x3e4a000 2026-02-11T23:31:17.439+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:11bee109:::obj319:head)]) -- 0x4911200 con 0x3e4a000 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.439+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1095 ==== MOSDPGPush(1.0 113/109 [PushOp(1:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3752000 2026-02-11T23:31:17.439+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.439+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1096 ==== pg_backfill(progress 1.0 e 113/113 lb 1:dc9e4e8c:::obj567:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3752000 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.439+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc9e4e8c:::obj567:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.439+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc9e4e8c:::obj567:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc9e4e8c:::obj567:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:dcf61aca:::obj332:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:dcf61aca:::obj332:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:dcf61aca:::obj332:head 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:dcf61aca:::obj332:head v 107'332 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.439+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dc9e4e8c:::obj567:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc9e4e8c:::obj567:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 113/113 lb 1:dc9e4e8c:::obj567:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:dc9e4e8c:::obj567:head) v3 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc7d10b2:::obj533:head local-lis/les=109/110 n=527 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:dc9e4e8c:::obj567:head) 2026-02-11T23:31:17.439+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dc9e4e8c:::obj567:head) v3 finish latency 0.000016 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.439+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.439+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d7b00 con 0x3752000 2026-02-11T23:31:17.439+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:dcf61aca:::obj332:head)]) -- 0x4d67b00 con 0x3752000 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.439+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 167 ==== MOSDPGPush(4.0 113/112 [PushOp(4:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2680 con 0x3e4a000 2026-02-11T23:31:17.439+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.439+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 168 ==== pg_backfill(progress 4.0 e 113/113 lb 4:11bee109:::obj319:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3e4a000 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.439+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:11bee109:::obj319:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.439+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:11bee109:::obj319:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 113/112 [PushOp(4:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.439+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:11bee109:::obj319:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.439+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:121cd428:::obj443:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.439+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:121cd428:::obj443:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:121cd428:::obj443:head 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:121cd428:::obj443:head v 107'443 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.440+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:121cd428:::obj443:head, version: 107'443, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:121cd428:::obj443:head@107'443, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:11bee109:::obj319:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:11bee109:::obj319:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 113/113 lb 4:11bee109:::obj319:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:11bee109:::obj319:head) v3 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11a85aa0:::obj43:head local-lis/les=112/113 n=60 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:11bee109:::obj319:head) 2026-02-11T23:31:17.440+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:11bee109:::obj319:head) v3 finish latency 0.000017 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.440+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49a3e00 con 0x3e4a000 2026-02-11T23:31:17.440+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:121cd428:::obj443:head)]) -- 0x4911380 con 0x3e4a000 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1097 ==== MOSDPGPush(1.0 113/109 [PushOp(1:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3752000 2026-02-11T23:31:17.440+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.440+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.440+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1098 ==== pg_backfill(progress 1.0 e 113/113 lb 1:dcf61aca:::obj332:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3752000 2026-02-11T23:31:17.440+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:dcf61aca:::obj332:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.440+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dcf61aca:::obj332:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 113/109 [PushOp(1:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dcf61aca:::obj332:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ddce93c9:::obj164:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ddce93c9:::obj164:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ddce93c9:::obj164:head 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ddce93c9:::obj164:head v 107'164 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.440+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dcf61aca:::obj332:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dcf61aca:::obj332:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 113/113 lb 1:dcf61aca:::obj332:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:dcf61aca:::obj332:head) v3 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dc9e4e8c:::obj567:head local-lis/les=109/110 n=528 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:dcf61aca:::obj332:head) 2026-02-11T23:31:17.440+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dcf61aca:::obj332:head) v3 finish latency 0.000017 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.440+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d6780 con 0x3752000 2026-02-11T23:31:17.440+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:ddce93c9:::obj164:head)]) -- 0x4d66600 con 0x3752000 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 169 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51824e0 con 0x3e4a000 2026-02-11T23:31:17.440+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.440+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 170 ==== pg_backfill(progress 4.0 e 113/113 lb 4:121cd428:::obj443:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3e4a000 2026-02-11T23:31:17.440+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:121cd428:::obj443:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.440+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:121cd428:::obj443:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 113/112 [PushOp(4:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:121cd428:::obj443:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1243fe1a:::obj21:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1243fe1a:::obj21:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1243fe1a:::obj21:head 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1243fe1a:::obj21:head v 107'21 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.440+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1243fe1a:::obj21:head, version: 107'21, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1243fe1a:::obj21:head@107'21, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:121cd428:::obj443:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:121cd428:::obj443:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 113/113 lb 4:121cd428:::obj443:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:121cd428:::obj443:head) v3 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:11bee109:::obj319:head local-lis/les=112/113 n=61 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:121cd428:::obj443:head) 2026-02-11T23:31:17.440+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:121cd428:::obj443:head) v3 finish latency 0.000018 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1099 ==== MOSDPGPush(1.0 113/109 [PushOp(1:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182000 con 0x3752000 2026-02-11T23:31:17.440+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.440+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1100 ==== pg_backfill(progress 1.0 e 113/113 lb 1:ddce93c9:::obj164:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3752000 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.440+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:ddce93c9:::obj164:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.440+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ddce93c9:::obj164:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ddce93c9:::obj164:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:de0491ba:::obj374:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:de0491ba:::obj374:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:de0491ba:::obj374:head 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:de0491ba:::obj374:head v 107'374 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.440+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.440+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ddce93c9:::obj164:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ddce93c9:::obj164:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 113/113 lb 1:ddce93c9:::obj164:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:ddce93c9:::obj164:head) v3 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dcf61aca:::obj332:head local-lis/les=109/110 n=529 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:ddce93c9:::obj164:head) 2026-02-11T23:31:17.440+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ddce93c9:::obj164:head) v3 finish latency 0.000014 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.440+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.440+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4916300 con 0x3e4a000 2026-02-11T23:31:17.440+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1243fe1a:::obj21:head)]) -- 0x4911500 con 0x3e4a000 2026-02-11T23:31:17.440+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.441+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d6600 con 0x3752000 2026-02-11T23:31:17.441+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:de0491ba:::obj374:head)]) -- 0x4916480 con 0x3752000 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.441+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 171 ==== MOSDPGPush(4.0 113/112 [PushOp(4:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516da00 con 0x3e4a000 2026-02-11T23:31:17.441+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.441+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.441+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 172 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1243fe1a:::obj21:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3e4a000 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.441+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1243fe1a:::obj21:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.441+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1243fe1a:::obj21:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 113/112 [PushOp(4:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1243fe1a:::obj21:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:124a99b2:::obj215:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:124a99b2:::obj215:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:124a99b2:::obj215:head 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:124a99b2:::obj215:head v 107'215 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.441+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.441+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1101 ==== MOSDPGPush(1.0 113/109 [PushOp(1:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4680 con 0x3752000 2026-02-11T23:31:17.441+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.441+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.441+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1102 ==== pg_backfill(progress 1.0 e 113/113 lb 1:de0491ba:::obj374:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3752000 2026-02-11T23:31:17.441+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:de0491ba:::obj374:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.441+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:de0491ba:::obj374:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:124a99b2:::obj215:head, version: 107'215, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:124a99b2:::obj215:head@107'215, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1243fe1a:::obj21:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1243fe1a:::obj21:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 113/113 lb 4:1243fe1a:::obj21:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 113/109 [PushOp(1:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1243fe1a:::obj21:head) v3 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:121cd428:::obj443:head local-lis/les=112/113 n=62 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1243fe1a:::obj21:head) 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.441+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1243fe1a:::obj21:head) v3 finish latency 0.000018 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:deffd1ec:::obj134:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:de0491ba:::obj374:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:deffd1ec:::obj134:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:deffd1ec:::obj134:head 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:deffd1ec:::obj134:head v 107'134 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.441+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.441+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:de0491ba:::obj374:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:de0491ba:::obj374:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 1.0 e 113/113 lb 1:de0491ba:::obj374:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:de0491ba:::obj374:head) v3 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ddce93c9:::obj164:head local-lis/les=109/110 n=530 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:de0491ba:::obj374:head) 2026-02-11T23:31:17.441+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:de0491ba:::obj374:head) v3 finish latency 0.000015 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.441+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4954300 con 0x3e4a000 2026-02-11T23:31:17.441+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:124a99b2:::obj215:head)]) -- 0x4dd6180 con 0x3e4a000 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.441+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.441+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.441+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d6180 con 0x3752000 2026-02-11T23:31:17.441+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:deffd1ec:::obj134:head)]) -- 0x4dd6480 con 0x3752000 2026-02-11T23:31:17.441+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.442+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 173 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d040 con 0x3e4a000 2026-02-11T23:31:17.442+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.442+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 174 ==== pg_backfill(progress 4.0 e 113/113 lb 4:124a99b2:::obj215:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3e4a000 2026-02-11T23:31:17.442+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:124a99b2:::obj215:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.442+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:124a99b2:::obj215:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:124a99b2:::obj215:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1405054b:::obj129:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1405054b:::obj129:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1405054b:::obj129:head 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1405054b:::obj129:head v 107'129 2026-02-11T23:31:17.442+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1103 ==== MOSDPGPush(1.0 113/109 [PushOp(1:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516cb60 con 0x3752000 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.442+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.442+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.442+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.442+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1104 ==== pg_backfill(progress 1.0 e 113/113 lb 1:deffd1ec:::obj134:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3752000 2026-02-11T23:31:17.442+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:deffd1ec:::obj134:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.442+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:deffd1ec:::obj134:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1405054b:::obj129:head, version: 107'129, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1405054b:::obj129:head@107'129, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000050 MOSDPGPush(1.0 113/109 [PushOp(1:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:124a99b2:::obj215:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:124a99b2:::obj215:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 113/113 lb 4:124a99b2:::obj215:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:124a99b2:::obj215:head) v3 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1243fe1a:::obj21:head local-lis/les=112/113 n=63 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:124a99b2:::obj215:head) 2026-02-11T23:31:17.442+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:dfa74a43:::obj27:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:dfa74a43:::obj27:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:dfa74a43:::obj27:head 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:124a99b2:::obj215:head) v3 finish latency 0.000026 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:dfa74a43:::obj27:head v 107'27 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:deffd1ec:::obj134:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.442+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:deffd1ec:::obj134:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:deffd1ec:::obj134:head) v3 prio 15 cost 0 latency 0.000105 pg_backfill(progress 1.0 e 113/113 lb 1:deffd1ec:::obj134:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:deffd1ec:::obj134:head) v3 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:de0491ba:::obj374:head local-lis/les=109/110 n=531 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:deffd1ec:::obj134:head) 2026-02-11T23:31:17.442+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:deffd1ec:::obj134:head) v3 finish latency 0.000014 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.442+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4917500 con 0x3e4a000 2026-02-11T23:31:17.442+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1405054b:::obj129:head)]) -- 0x4dd6780 con 0x3e4a000 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.442+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49b9680 con 0x3752000 2026-02-11T23:31:17.442+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:dfa74a43:::obj27:head)]) -- 0x4dd6a80 con 0x3752000 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.442+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 175 ==== MOSDPGPush(4.0 113/112 [PushOp(4:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.442+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.442+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 176 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1405054b:::obj129:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.442+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1405054b:::obj129:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.442+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1405054b:::obj129:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 113/112 [PushOp(4:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1405054b:::obj129:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:147354e0:::obj436:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:147354e0:::obj436:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:147354e0:::obj436:head 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:147354e0:::obj436:head v 107'436 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.442+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.442+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1105 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3752000 2026-02-11T23:31:17.442+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.442+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.442+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1106 ==== pg_backfill(progress 1.0 e 113/113 lb 1:dfa74a43:::obj27:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164100 con 0x3752000 2026-02-11T23:31:17.442+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:dfa74a43:::obj27:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.442+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dfa74a43:::obj27:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:147354e0:::obj436:head, version: 107'436, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147354e0:::obj436:head@107'436, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(1.0 113/109 [PushOp(1:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1405054b:::obj129:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1405054b:::obj129:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 113/113 lb 4:1405054b:::obj129:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1405054b:::obj129:head) v3 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:124a99b2:::obj215:head local-lis/les=112/113 n=64 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1405054b:::obj129:head) 2026-02-11T23:31:17.442+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e03f6950:::obj575:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1405054b:::obj129:head) v3 finish latency 0.000019 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e03f6950:::obj575:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e03f6950:::obj575:head 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dfa74a43:::obj27:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e03f6950:::obj575:head v 107'575 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.442+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.442+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.442+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:dfa74a43:::obj27:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dfa74a43:::obj27:head) v3 prio 15 cost 0 latency 0.000103 pg_backfill(progress 1.0 e 113/113 lb 1:dfa74a43:::obj27:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:dfa74a43:::obj27:head) v3 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:deffd1ec:::obj134:head local-lis/les=109/110 n=532 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:dfa74a43:::obj27:head) 2026-02-11T23:31:17.442+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.442+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:dfa74a43:::obj27:head) v3 finish latency 0.000015 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.442+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.443+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x497e000 con 0x3e4a000 2026-02-11T23:31:17.443+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:147354e0:::obj436:head)]) -- 0x4dd6c00 con 0x3e4a000 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.443+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49b9380 con 0x3752000 2026-02-11T23:31:17.443+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e03f6950:::obj575:head)]) -- 0x4dd6f00 con 0x3752000 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.443+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1107 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3752000 2026-02-11T23:31:17.443+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.443+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.443+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1108 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e03f6950:::obj575:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3752000 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.443+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e03f6950:::obj575:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.443+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e03f6950:::obj575:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e03f6950:::obj575:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e05d79f1:::obj206:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e05d79f1:::obj206:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e05d79f1:::obj206:head 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e05d79f1:::obj206:head v 107'206 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.443+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.443+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 177 ==== MOSDPGPush(4.0 113/112 [PushOp(4:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3e4a000 2026-02-11T23:31:17.443+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.443+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.443+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 178 ==== pg_backfill(progress 4.0 e 113/113 lb 4:147354e0:::obj436:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.443+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:147354e0:::obj436:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.443+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:147354e0:::obj436:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e03f6950:::obj575:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e03f6950:::obj575:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 113/113 lb 1:e03f6950:::obj575:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e03f6950:::obj575:head) v3 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 113/112 [PushOp(4:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:dfa74a43:::obj27:head local-lis/les=109/110 n=533 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e03f6950:::obj575:head) 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.443+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e03f6950:::obj575:head) v3 finish latency 0.000017 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:147b5010:::obj178:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:147b5010:::obj178:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:147354e0:::obj436:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:147b5010:::obj178:head 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:147b5010:::obj178:head v 107'178 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.443+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.443+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:147b5010:::obj178:head, version: 107'178, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:147b5010:::obj178:head@107'178, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:147354e0:::obj436:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:147354e0:::obj436:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 113/113 lb 4:147354e0:::obj436:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:147354e0:::obj436:head) v3 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1405054b:::obj129:head local-lis/les=112/113 n=65 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:147354e0:::obj436:head) 2026-02-11T23:31:17.443+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:147354e0:::obj436:head) v3 finish latency 0.000015 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.443+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4917c80 con 0x3752000 2026-02-11T23:31:17.443+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e05d79f1:::obj206:head)]) -- 0x4dd7200 con 0x3752000 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.443+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.443+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.443+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49b9080 con 0x3e4a000 2026-02-11T23:31:17.443+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:147b5010:::obj178:head)]) -- 0x4dd7500 con 0x3e4a000 2026-02-11T23:31:17.443+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.444+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1109 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a3860 con 0x3752000 2026-02-11T23:31:17.444+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.444+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1110 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e05d79f1:::obj206:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3752000 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.444+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e05d79f1:::obj206:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.444+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e05d79f1:::obj206:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e05d79f1:::obj206:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e0a44829:::obj14:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e0a44829:::obj14:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e0a44829:::obj14:head 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e0a44829:::obj14:head v 107'14 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.444+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e05d79f1:::obj206:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e05d79f1:::obj206:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 113/113 lb 1:e05d79f1:::obj206:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e05d79f1:::obj206:head) v3 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e03f6950:::obj575:head local-lis/les=109/110 n=534 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e05d79f1:::obj206:head) 2026-02-11T23:31:17.444+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e05d79f1:::obj206:head) v3 finish latency 0.000016 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.444+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4910600 con 0x3752000 2026-02-11T23:31:17.444+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e0a44829:::obj14:head)]) -- 0x4dd7800 con 0x3752000 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.444+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 179 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.444+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.444+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 180 ==== pg_backfill(progress 4.0 e 113/113 lb 4:147b5010:::obj178:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.444+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:147b5010:::obj178:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.444+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:147b5010:::obj178:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 113/112 [PushOp(4:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:147b5010:::obj178:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1503467e:::obj118:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1503467e:::obj118:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1503467e:::obj118:head 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1503467e:::obj118:head v 107'118 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.444+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1503467e:::obj118:head, version: 107'118, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1503467e:::obj118:head@107'118, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:147b5010:::obj178:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:147b5010:::obj178:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 113/113 lb 4:147b5010:::obj178:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:147b5010:::obj178:head) v3 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147354e0:::obj436:head local-lis/les=112/113 n=66 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:147b5010:::obj178:head) 2026-02-11T23:31:17.444+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:147b5010:::obj178:head) v3 finish latency 0.000016 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.444+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1111 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3752000 2026-02-11T23:31:17.444+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.444+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.444+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49b8f00 con 0x3e4a000 2026-02-11T23:31:17.444+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1503467e:::obj118:head)]) -- 0x4dd7980 con 0x3e4a000 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1112 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e0a44829:::obj14:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f6680 con 0x3752000 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.444+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e0a44829:::obj14:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.444+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e0a44829:::obj14:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e0a44829:::obj14:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(1.0 113/109 [PushOp(1:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e10533c6:::obj450:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e10533c6:::obj450:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e10533c6:::obj450:head 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e10533c6:::obj450:head v 107'450 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.444+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.444+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e0a44829:::obj14:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e0a44829:::obj14:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 113/113 lb 1:e0a44829:::obj14:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e0a44829:::obj14:head) v3 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e05d79f1:::obj206:head local-lis/les=109/110 n=535 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e0a44829:::obj14:head) 2026-02-11T23:31:17.444+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e0a44829:::obj14:head) v3 finish latency 0.000017 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.444+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.444+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.444+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497e480 con 0x3752000 2026-02-11T23:31:17.444+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e10533c6:::obj450:head)]) -- 0x4dd7c80 con 0x3752000 2026-02-11T23:31:17.444+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 181 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.445+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.445+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 182 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1503467e:::obj118:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.445+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1503467e:::obj118:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1503467e:::obj118:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 113/112 [PushOp(4:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1503467e:::obj118:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1547e7aa:::obj466:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1547e7aa:::obj466:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1547e7aa:::obj466:head 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1547e7aa:::obj466:head v 107'466 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.445+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1547e7aa:::obj466:head, version: 107'466, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1547e7aa:::obj466:head@107'466, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1503467e:::obj118:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1503467e:::obj118:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 113/113 lb 4:1503467e:::obj118:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1503467e:::obj118:head) v3 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:147b5010:::obj178:head local-lis/les=112/113 n=67 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1503467e:::obj118:head) 2026-02-11T23:31:17.445+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1503467e:::obj118:head) v3 finish latency 0.000016 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.445+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49b8c00 con 0x3e4a000 2026-02-11T23:31:17.445+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1547e7aa:::obj466:head)]) -- 0x49d6180 con 0x3e4a000 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1113 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3752000 2026-02-11T23:31:17.445+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.445+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 113/109 [PushOp(1:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.445+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1114 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e10533c6:::obj450:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3752000 2026-02-11T23:31:17.445+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e10533c6:::obj450:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.445+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e10533c6:::obj450:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e10533c6:::obj450:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e196274b:::obj304:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e196274b:::obj304:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e196274b:::obj304:head 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e196274b:::obj304:head v 107'304 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.445+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e10533c6:::obj450:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e10533c6:::obj450:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 113/113 lb 1:e10533c6:::obj450:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e10533c6:::obj450:head) v3 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e0a44829:::obj14:head local-lis/les=109/110 n=536 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e10533c6:::obj450:head) 2026-02-11T23:31:17.445+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e10533c6:::obj450:head) v3 finish latency 0.000015 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.445+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x497e180 con 0x3752000 2026-02-11T23:31:17.445+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e196274b:::obj304:head)]) -- 0x4954300 con 0x3752000 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 183 ==== MOSDPGPush(4.0 113/112 [PushOp(4:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3e4a000 2026-02-11T23:31:17.445+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.445+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 184 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1547e7aa:::obj466:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3e4a000 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.445+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1547e7aa:::obj466:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.445+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1547e7aa:::obj466:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1547e7aa:::obj466:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 113/112 [PushOp(4:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:170a838d:::obj199:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:170a838d:::obj199:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:170a838d:::obj199:head 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:170a838d:::obj199:head v 107'199 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.445+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:170a838d:::obj199:head, version: 107'199, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:170a838d:::obj199:head@107'199, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1547e7aa:::obj466:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1547e7aa:::obj466:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 113/113 lb 4:1547e7aa:::obj466:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1547e7aa:::obj466:head) v3 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1503467e:::obj118:head local-lis/les=112/113 n=68 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1547e7aa:::obj466:head) 2026-02-11T23:31:17.445+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1547e7aa:::obj466:head) v3 finish latency 0.000018 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1115 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3752000 2026-02-11T23:31:17.445+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.445+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1116 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e196274b:::obj304:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165180 con 0x3752000 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.445+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e196274b:::obj304:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.445+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e196274b:::obj304:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 113/109 [PushOp(1:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e196274b:::obj304:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e1d56785:::obj141:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e1d56785:::obj141:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e1d56785:::obj141:head 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e1d56785:::obj141:head v 107'141 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.445+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.445+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.445+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e196274b:::obj304:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e196274b:::obj304:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 1.0 e 113/113 lb 1:e196274b:::obj304:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e196274b:::obj304:head) v3 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e10533c6:::obj450:head local-lis/les=109/110 n=537 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e196274b:::obj304:head) 2026-02-11T23:31:17.445+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e196274b:::obj304:head) v3 finish latency 0.000018 2026-02-11T23:31:17.445+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.445+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49b8900 con 0x3e4a000 2026-02-11T23:31:17.445+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:170a838d:::obj199:head)]) -- 0x49d6600 con 0x3e4a000 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.445+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.446+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4910000 con 0x3752000 2026-02-11T23:31:17.446+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e1d56785:::obj141:head)]) -- 0x4916300 con 0x3752000 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.446+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 185 ==== MOSDPGPush(4.0 113/112 [PushOp(4:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.446+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.446+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.446+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 186 ==== pg_backfill(progress 4.0 e 113/113 lb 4:170a838d:::obj199:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164c00 con 0x3e4a000 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.446+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:170a838d:::obj199:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.446+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:170a838d:::obj199:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:170a838d:::obj199:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 113/112 [PushOp(4:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.446+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1117 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d6c0 con 0x3752000 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.446+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.446+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:171b715e:::obj570:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:171b715e:::obj570:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.446+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1118 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e1d56785:::obj141:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3752000 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:171b715e:::obj570:head 2026-02-11T23:31:17.446+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e1d56785:::obj141:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:171b715e:::obj570:head v 107'570 2026-02-11T23:31:17.446+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e1d56785:::obj141:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.446+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:171b715e:::obj570:head, version: 107'570, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:171b715e:::obj570:head@107'570, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:170a838d:::obj199:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:170a838d:::obj199:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 113/113 lb 4:170a838d:::obj199:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000059 MOSDPGPush(1.0 113/109 [PushOp(1:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:170a838d:::obj199:head) v3 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1547e7aa:::obj466:head local-lis/les=112/113 n=69 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:170a838d:::obj199:head) 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.446+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e25e0189:::obj579:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e25e0189:::obj579:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:170a838d:::obj199:head) v3 finish latency 0.000020 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e25e0189:::obj579:head 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e25e0189:::obj579:head v 107'579 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e1d56785:::obj141:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.446+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.446+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e1d56785:::obj141:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e1d56785:::obj141:head) v3 prio 15 cost 0 latency 0.000109 pg_backfill(progress 1.0 e 113/113 lb 1:e1d56785:::obj141:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e1d56785:::obj141:head) v3 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e196274b:::obj304:head local-lis/les=109/110 n=538 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e1d56785:::obj141:head) 2026-02-11T23:31:17.446+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e1d56785:::obj141:head) v3 finish latency 0.000015 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.446+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49b8780 con 0x3e4a000 2026-02-11T23:31:17.446+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:171b715e:::obj570:head)]) -- 0x49d6780 con 0x3e4a000 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.446+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.446+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4910180 con 0x3752000 2026-02-11T23:31:17.446+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e25e0189:::obj579:head)]) -- 0x49a3e00 con 0x3752000 2026-02-11T23:31:17.446+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.446+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.447+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 187 ==== MOSDPGPush(4.0 113/112 [PushOp(4:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.447+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.447+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 188 ==== pg_backfill(progress 4.0 e 113/113 lb 4:171b715e:::obj570:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def180 con 0x3e4a000 2026-02-11T23:31:17.447+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:171b715e:::obj570:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.447+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:171b715e:::obj570:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:171b715e:::obj570:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 113/112 [PushOp(4:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.447+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1119 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cd00 con 0x3752000 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.447+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000005 epoch 113 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.447+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.447+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1120 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e25e0189:::obj579:head) ==== 1062+0+0 (unknown 0 0 0) 0x4deec00 con 0x3752000 2026-02-11T23:31:17.447+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e25e0189:::obj579:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:17371e94:::obj362:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.447+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e25e0189:::obj579:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:17371e94:::obj362:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:17371e94:::obj362:head 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:17371e94:::obj362:head v 107'362 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.447+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:17371e94:::obj362:head, version: 107'362, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:17371e94:::obj362:head@107'362, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:171b715e:::obj570:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:171b715e:::obj570:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 113/113 lb 4:171b715e:::obj570:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000063 MOSDPGPush(1.0 113/109 [PushOp(1:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:171b715e:::obj570:head) v3 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:170a838d:::obj199:head local-lis/les=112/113 n=70 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:171b715e:::obj570:head) 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.447+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e26151b8:::obj492:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:171b715e:::obj570:head) v3 finish latency 0.000020 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e26151b8:::obj492:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e26151b8:::obj492:head 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e26151b8:::obj492:head v 107'492 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e25e0189:::obj579:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.447+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000101 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e25e0189:::obj579:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e25e0189:::obj579:head) v3 prio 15 cost 0 latency 0.000162 pg_backfill(progress 1.0 e 113/113 lb 1:e25e0189:::obj579:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e25e0189:::obj579:head) v3 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e1d56785:::obj141:head local-lis/les=109/110 n=539 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e25e0189:::obj579:head) 2026-02-11T23:31:17.447+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e25e0189:::obj579:head) v3 finish latency 0.000017 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.447+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4a04900 con 0x3e4a000 2026-02-11T23:31:17.447+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:17371e94:::obj362:head)]) -- 0x49d7b00 con 0x3e4a000 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.447+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4eabc80 con 0x3752000 2026-02-11T23:31:17.447+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e26151b8:::obj492:head)]) -- 0x49d7e00 con 0x3752000 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.447+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 189 ==== MOSDPGPush(4.0 113/112 [PushOp(4:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3e4a000 2026-02-11T23:31:17.447+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.447+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 190 ==== pg_backfill(progress 4.0 e 113/113 lb 4:17371e94:::obj362:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3e4a000 2026-02-11T23:31:17.447+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:17371e94:::obj362:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.447+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:17371e94:::obj362:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:17371e94:::obj362:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 113/112 [PushOp(4:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:18100969:::obj255:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:18100969:::obj255:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:18100969:::obj255:head 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:18100969:::obj255:head v 107'255 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.447+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:18100969:::obj255:head, version: 107'255, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18100969:::obj255:head@107'255, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:17371e94:::obj362:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:17371e94:::obj362:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 113/113 lb 4:17371e94:::obj362:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:17371e94:::obj362:head) v3 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:171b715e:::obj570:head local-lis/les=112/113 n=71 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:17371e94:::obj362:head) 2026-02-11T23:31:17.447+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:17371e94:::obj362:head) v3 finish latency 0.000018 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.447+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1121 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51831e0 con 0x3752000 2026-02-11T23:31:17.447+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.447+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1122 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e26151b8:::obj492:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3752000 2026-02-11T23:31:17.447+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e26151b8:::obj492:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.447+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e26151b8:::obj492:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e26151b8:::obj492:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 113/109 [PushOp(1:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e3cd6c93:::obj259:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e3cd6c93:::obj259:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e3cd6c93:::obj259:head 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e3cd6c93:::obj259:head v 107'259 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.447+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.447+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.447+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e26151b8:::obj492:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e26151b8:::obj492:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 1.0 e 113/113 lb 1:e26151b8:::obj492:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e26151b8:::obj492:head) v3 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e25e0189:::obj579:head local-lis/les=109/110 n=540 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e26151b8:::obj492:head) 2026-02-11T23:31:17.447+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e26151b8:::obj492:head) v3 finish latency 0.000032 2026-02-11T23:31:17.447+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.447+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4a04c00 con 0x3e4a000 2026-02-11T23:31:17.447+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:18100969:::obj255:head)]) -- 0x49d7980 con 0x3e4a000 2026-02-11T23:31:17.447+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.448+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4911c80 con 0x3752000 2026-02-11T23:31:17.448+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e3cd6c93:::obj259:head)]) -- 0x49d7500 con 0x3752000 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.448+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1123 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3752000 2026-02-11T23:31:17.448+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.448+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.448+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1124 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e3cd6c93:::obj259:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3752000 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.448+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e3cd6c93:::obj259:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.448+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e3cd6c93:::obj259:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e3cd6c93:::obj259:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.448+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 191 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51821a0 con 0x3e4a000 2026-02-11T23:31:17.448+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e4109927:::obj482:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e4109927:::obj482:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.448+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e4109927:::obj482:head 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e4109927:::obj482:head v 107'482 2026-02-11T23:31:17.448+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 192 ==== pg_backfill(progress 4.0 e 113/113 lb 4:18100969:::obj255:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded080 con 0x3e4a000 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.448+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:18100969:::obj255:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.448+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:18100969:::obj255:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.448+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e3cd6c93:::obj259:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000066 MOSDPGPush(4.0 113/112 [PushOp(4:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e3cd6c93:::obj259:head) v3 prio 15 cost 0 latency 0.000104 pg_backfill(progress 1.0 e 113/113 lb 1:e3cd6c93:::obj259:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e3cd6c93:::obj259:head) v3 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e26151b8:::obj492:head local-lis/les=109/110 n=541 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e3cd6c93:::obj259:head) 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.448+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1842fa44:::obj284:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1842fa44:::obj284:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e3cd6c93:::obj259:head) v3 finish latency 0.000022 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1842fa44:::obj284:head 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1842fa44:::obj284:head v 107'284 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:18100969:::obj255:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.448+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1842fa44:::obj284:head, version: 107'284, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1842fa44:::obj284:head@107'284, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:18100969:::obj255:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:18100969:::obj255:head) v3 prio 15 cost 0 latency 0.000123 pg_backfill(progress 4.0 e 113/113 lb 4:18100969:::obj255:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:18100969:::obj255:head) v3 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:17371e94:::obj362:head local-lis/les=112/113 n=72 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:18100969:::obj255:head) 2026-02-11T23:31:17.448+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:18100969:::obj255:head) v3 finish latency 0.000016 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.448+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4a04780 con 0x3752000 2026-02-11T23:31:17.448+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e4109927:::obj482:head)]) -- 0x49d7800 con 0x3752000 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.448+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.448+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4eab080 con 0x3e4a000 2026-02-11T23:31:17.448+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1842fa44:::obj284:head)]) -- 0x49d7200 con 0x3e4a000 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.448+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1125 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3752000 2026-02-11T23:31:17.448+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.448+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.448+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1126 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e4109927:::obj482:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3752000 2026-02-11T23:31:17.448+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e4109927:::obj482:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.448+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e4109927:::obj482:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.448+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.448+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e4109927:::obj482:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 113/109 [PushOp(1:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.448+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e43ffde6:::obj285:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e43ffde6:::obj285:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e43ffde6:::obj285:head 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e43ffde6:::obj285:head v 107'285 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.449+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e4109927:::obj482:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e4109927:::obj482:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 1.0 e 113/113 lb 1:e4109927:::obj482:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e4109927:::obj482:head) v3 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e3cd6c93:::obj259:head local-lis/les=109/110 n=542 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e4109927:::obj482:head) 2026-02-11T23:31:17.449+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e4109927:::obj482:head) v3 finish latency 0.000022 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.449+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4a04600 con 0x3752000 2026-02-11T23:31:17.449+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e43ffde6:::obj285:head)]) -- 0x49d7380 con 0x3752000 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 193 ==== MOSDPGPush(4.0 113/112 [PushOp(4:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3e4a000 2026-02-11T23:31:17.449+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.449+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 194 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1842fa44:::obj284:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3e4a000 2026-02-11T23:31:17.449+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1842fa44:::obj284:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.449+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1842fa44:::obj284:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1842fa44:::obj284:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 113/112 [PushOp(4:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:18b572c6:::obj440:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:18b572c6:::obj440:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:18b572c6:::obj440:head 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:18b572c6:::obj440:head v 107'440 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.449+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:18b572c6:::obj440:head, version: 107'440, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18b572c6:::obj440:head@107'440, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1842fa44:::obj284:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1842fa44:::obj284:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 113/113 lb 4:1842fa44:::obj284:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1842fa44:::obj284:head) v3 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18100969:::obj255:head local-lis/les=112/113 n=73 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1842fa44:::obj284:head) 2026-02-11T23:31:17.449+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1842fa44:::obj284:head) v3 finish latency 0.000026 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.449+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4911b00 con 0x3e4a000 2026-02-11T23:31:17.449+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:18b572c6:::obj440:head)]) -- 0x49d7080 con 0x3e4a000 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1127 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x389b040 con 0x3752000 2026-02-11T23:31:17.449+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.449+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1128 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e43ffde6:::obj285:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3752000 2026-02-11T23:31:17.449+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e43ffde6:::obj285:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.449+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e43ffde6:::obj285:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(1.0 113/109 [PushOp(1:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e43ffde6:::obj285:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e46d208d:::obj89:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e46d208d:::obj89:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e46d208d:::obj89:head 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e46d208d:::obj89:head v 107'89 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.449+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e43ffde6:::obj285:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e43ffde6:::obj285:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 1.0 e 113/113 lb 1:e43ffde6:::obj285:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e43ffde6:::obj285:head) v3 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e4109927:::obj482:head local-lis/les=109/110 n=543 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e43ffde6:::obj285:head) 2026-02-11T23:31:17.449+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e43ffde6:::obj285:head) v3 finish latency 0.000027 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.449+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4a04480 con 0x3752000 2026-02-11T23:31:17.449+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e46d208d:::obj89:head)]) -- 0x49d6c00 con 0x3752000 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 195 ==== MOSDPGPush(4.0 113/112 [PushOp(4:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516d6c0 con 0x3e4a000 2026-02-11T23:31:17.449+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.449+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 196 ==== pg_backfill(progress 4.0 e 113/113 lb 4:18b572c6:::obj440:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3e4a000 2026-02-11T23:31:17.449+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:18b572c6:::obj440:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.449+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:18b572c6:::obj440:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:18b572c6:::obj440:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 113/112 [PushOp(4:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:18c50d58:::obj63:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:18c50d58:::obj63:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:18c50d58:::obj63:head 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:18c50d58:::obj63:head v 107'63 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.449+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.449+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:18c50d58:::obj63:head, version: 107'63, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:18c50d58:::obj63:head@107'63, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.449+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.449+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:18b572c6:::obj440:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:18b572c6:::obj440:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 113/113 lb 4:18b572c6:::obj440:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:18b572c6:::obj440:head) v3 2026-02-11T23:31:17.449+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1842fa44:::obj284:head local-lis/les=112/113 n=74 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:18b572c6:::obj440:head) 2026-02-11T23:31:17.450+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:18b572c6:::obj440:head) v3 finish latency 0.000020 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.450+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1129 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3752000 2026-02-11T23:31:17.450+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.450+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1130 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e46d208d:::obj89:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3752000 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.450+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e46d208d:::obj89:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.450+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e46d208d:::obj89:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 113/109 [PushOp(1:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e46d208d:::obj89:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e48b6d56:::obj389:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e48b6d56:::obj389:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e48b6d56:::obj389:head 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e48b6d56:::obj389:head v 107'389 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.450+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e46d208d:::obj89:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e46d208d:::obj89:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 1.0 e 113/113 lb 1:e46d208d:::obj89:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e46d208d:::obj89:head) v3 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e43ffde6:::obj285:head local-lis/les=109/110 n=544 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e46d208d:::obj89:head) 2026-02-11T23:31:17.450+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e46d208d:::obj89:head) v3 finish latency 0.000017 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.450+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4eaa780 con 0x3e4a000 2026-02-11T23:31:17.450+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:18c50d58:::obj63:head)]) -- 0x4916000 con 0x3e4a000 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.450+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4a04180 con 0x3752000 2026-02-11T23:31:17.450+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e48b6d56:::obj389:head)]) -- 0x49d6f00 con 0x3752000 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.450+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 197 ==== MOSDPGPush(4.0 113/112 [PushOp(4:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3e4a000 2026-02-11T23:31:17.450+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.450+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 198 ==== pg_backfill(progress 4.0 e 113/113 lb 4:18c50d58:::obj63:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3e4a000 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.450+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:18c50d58:::obj63:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.450+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:18c50d58:::obj63:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 113/112 [PushOp(4:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:18c50d58:::obj63:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:19299046:::obj541:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.450+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1131 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3752000 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:19299046:::obj541:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:19299046:::obj541:head 2026-02-11T23:31:17.450+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:19299046:::obj541:head v 107'541 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.450+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.450+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.450+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1132 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e48b6d56:::obj389:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3752000 2026-02-11T23:31:17.450+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e48b6d56:::obj389:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.450+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e48b6d56:::obj389:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:19299046:::obj541:head, version: 107'541, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:19299046:::obj541:head@107'541, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:18c50d58:::obj63:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000059 MOSDPGPush(1.0 113/109 [PushOp(1:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:18c50d58:::obj63:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 113/113 lb 4:18c50d58:::obj63:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:18c50d58:::obj63:head) v3 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18b572c6:::obj440:head local-lis/les=112/113 n=75 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:18c50d58:::obj63:head) 2026-02-11T23:31:17.450+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:18c50d58:::obj63:head) v3 finish latency 0.000018 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e5124c98:::obj133:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e48b6d56:::obj389:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e5124c98:::obj133:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e5124c98:::obj133:head 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e5124c98:::obj133:head v 107'133 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.450+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.450+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e48b6d56:::obj389:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e48b6d56:::obj389:head) v3 prio 15 cost 0 latency 0.000114 pg_backfill(progress 1.0 e 113/113 lb 1:e48b6d56:::obj389:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e48b6d56:::obj389:head) v3 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e46d208d:::obj89:head local-lis/les=109/110 n=545 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e48b6d56:::obj389:head) 2026-02-11T23:31:17.450+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e48b6d56:::obj389:head) v3 finish latency 0.000016 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.450+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.450+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.450+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4911980 con 0x3e4a000 2026-02-11T23:31:17.450+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:19299046:::obj541:head)]) -- 0x4a04900 con 0x3e4a000 2026-02-11T23:31:17.450+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.451+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fce000 con 0x3752000 2026-02-11T23:31:17.451+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e5124c98:::obj133:head)]) -- 0x4910180 con 0x3752000 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.451+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1133 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3752000 2026-02-11T23:31:17.451+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.451+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.451+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1134 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e5124c98:::obj133:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3752000 2026-02-11T23:31:17.451+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e5124c98:::obj133:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.451+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e5124c98:::obj133:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 113/109 [PushOp(1:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e5124c98:::obj133:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e6508d05:::obj595:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e6508d05:::obj595:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e6508d05:::obj595:head 2026-02-11T23:31:17.451+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 199 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e6508d05:::obj595:head v 107'595 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.451+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.451+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.451+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.451+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 200 ==== pg_backfill(progress 4.0 e 113/113 lb 4:19299046:::obj541:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3e4a000 2026-02-11T23:31:17.451+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:19299046:::obj541:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.451+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:19299046:::obj541:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e5124c98:::obj133:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000055 MOSDPGPush(4.0 113/112 [PushOp(4:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e5124c98:::obj133:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 113/113 lb 1:e5124c98:::obj133:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e5124c98:::obj133:head) v3 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e48b6d56:::obj389:head local-lis/les=109/110 n=546 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e5124c98:::obj133:head) 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.451+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1a1150ce:::obj60:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1a1150ce:::obj60:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1a1150ce:::obj60:head 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1a1150ce:::obj60:head v 107'60 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e5124c98:::obj133:head) v3 finish latency 0.000028 2026-02-11T23:31:17.451+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:19299046:::obj541:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a1150ce:::obj60:head, version: 107'60, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a1150ce:::obj60:head@107'60, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:19299046:::obj541:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:19299046:::obj541:head) v3 prio 15 cost 0 latency 0.000110 pg_backfill(progress 4.0 e 113/113 lb 4:19299046:::obj541:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:19299046:::obj541:head) v3 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:18c50d58:::obj63:head local-lis/les=112/113 n=76 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:19299046:::obj541:head) 2026-02-11T23:31:17.451+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:19299046:::obj541:head) v3 finish latency 0.000014 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.451+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fce180 con 0x3752000 2026-02-11T23:31:17.451+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e6508d05:::obj595:head)]) -- 0x49b8780 con 0x3752000 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.451+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.451+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4fce300 con 0x3e4a000 2026-02-11T23:31:17.451+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1a1150ce:::obj60:head)]) -- 0x4910000 con 0x3e4a000 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.451+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1135 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3752000 2026-02-11T23:31:17.451+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.451+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.451+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1136 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e6508d05:::obj595:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3752000 2026-02-11T23:31:17.451+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e6508d05:::obj595:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.451+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e6508d05:::obj595:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.451+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 113/109 [PushOp(1:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.451+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e6508d05:::obj595:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.451+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e6e28dcd:::obj87:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e6e28dcd:::obj87:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e6e28dcd:::obj87:head 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e6e28dcd:::obj87:head v 107'87 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.452+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e6508d05:::obj595:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e6508d05:::obj595:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 1.0 e 113/113 lb 1:e6508d05:::obj595:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e6508d05:::obj595:head) v3 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e5124c98:::obj133:head local-lis/les=109/110 n=547 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e6508d05:::obj595:head) 2026-02-11T23:31:17.452+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e6508d05:::obj595:head) v3 finish latency 0.000026 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.452+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fce480 con 0x3752000 2026-02-11T23:31:17.452+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e6e28dcd:::obj87:head)]) -- 0x49d6600 con 0x3752000 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 201 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.452+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.452+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 202 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1a1150ce:::obj60:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5080 con 0x3e4a000 2026-02-11T23:31:17.452+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a1150ce:::obj60:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.452+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a1150ce:::obj60:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a1150ce:::obj60:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 113/112 [PushOp(4:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1a20990d:::obj103:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1a20990d:::obj103:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1a20990d:::obj103:head 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1a20990d:::obj103:head v 107'103 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.452+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a20990d:::obj103:head, version: 107'103, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a20990d:::obj103:head@107'103, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000087 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a1150ce:::obj60:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a1150ce:::obj60:head) v3 prio 15 cost 0 latency 0.000108 pg_backfill(progress 4.0 e 113/113 lb 4:1a1150ce:::obj60:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1a1150ce:::obj60:head) v3 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:19299046:::obj541:head local-lis/les=112/113 n=77 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1a1150ce:::obj60:head) 2026-02-11T23:31:17.452+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a1150ce:::obj60:head) v3 finish latency 0.000034 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.452+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4fce600 con 0x3e4a000 2026-02-11T23:31:17.452+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1a20990d:::obj103:head)]) -- 0x49b8900 con 0x3e4a000 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1137 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3752000 2026-02-11T23:31:17.452+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.452+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.452+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1138 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e6e28dcd:::obj87:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f4000 con 0x3752000 2026-02-11T23:31:17.452+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e6e28dcd:::obj87:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 113 2026-02-11T23:31:17.452+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e6e28dcd:::obj87:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 113/109 [PushOp(1:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e6e28dcd:::obj87:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e74a4ee3:::obj396:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e74a4ee3:::obj396:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e74a4ee3:::obj396:head 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e74a4ee3:::obj396:head v 107'396 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.452+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e6e28dcd:::obj87:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e6e28dcd:::obj87:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 113/113 lb 1:e6e28dcd:::obj87:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e6e28dcd:::obj87:head) v3 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6508d05:::obj595:head local-lis/les=109/110 n=548 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e6e28dcd:::obj87:head) 2026-02-11T23:31:17.452+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e6e28dcd:::obj87:head) v3 finish latency 0.000022 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.452+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fce780 con 0x3752000 2026-02-11T23:31:17.452+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e74a4ee3:::obj396:head)]) -- 0x497e180 con 0x3752000 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 203 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3e4a000 2026-02-11T23:31:17.452+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.452+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.452+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 204 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1a20990d:::obj103:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a20990d:::obj103:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.452+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a20990d:::obj103:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 113/112 [PushOp(4:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a20990d:::obj103:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1a2eef70:::obj126:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1a2eef70:::obj126:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1a2eef70:::obj126:head 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1a2eef70:::obj126:head v 107'126 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.452+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.452+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a2eef70:::obj126:head, version: 107'126, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a2eef70:::obj126:head@107'126, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000079 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.452+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.452+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a20990d:::obj103:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a20990d:::obj103:head) v3 prio 15 cost 0 latency 0.000101 pg_backfill(progress 4.0 e 113/113 lb 4:1a20990d:::obj103:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1a20990d:::obj103:head) v3 2026-02-11T23:31:17.452+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a1150ce:::obj60:head local-lis/les=112/113 n=78 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1a20990d:::obj103:head) 2026-02-11T23:31:17.452+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a20990d:::obj103:head) v3 finish latency 0.000024 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1139 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3752000 2026-02-11T23:31:17.453+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.453+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1140 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e74a4ee3:::obj396:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5600 con 0x3752000 2026-02-11T23:31:17.453+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e74a4ee3:::obj396:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.453+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e74a4ee3:::obj396:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 113/109 [PushOp(1:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e74a4ee3:::obj396:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e7b9c028:::obj510:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e7b9c028:::obj510:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e7b9c028:::obj510:head 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e7b9c028:::obj510:head v 107'510 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.453+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e74a4ee3:::obj396:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e74a4ee3:::obj396:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 1.0 e 113/113 lb 1:e74a4ee3:::obj396:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e74a4ee3:::obj396:head) v3 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e6e28dcd:::obj87:head local-lis/les=109/110 n=549 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e74a4ee3:::obj396:head) 2026-02-11T23:31:17.453+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e74a4ee3:::obj396:head) v3 finish latency 0.000021 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.453+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4fcec00 con 0x3e4a000 2026-02-11T23:31:17.453+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1a2eef70:::obj126:head)]) -- 0x4fce900 con 0x3e4a000 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.453+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fced80 con 0x3752000 2026-02-11T23:31:17.453+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e7b9c028:::obj510:head)]) -- 0x49b8c00 con 0x3752000 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1141 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3752000 2026-02-11T23:31:17.453+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.453+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1142 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e7b9c028:::obj510:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3752000 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.453+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e7b9c028:::obj510:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 113 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e7b9c028:::obj510:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 113/109 [PushOp(1:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e7b9c028:::obj510:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e7f87738:::obj356:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e7f87738:::obj356:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e7f87738:::obj356:head 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e7f87738:::obj356:head v 107'356 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.453+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.453+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 205 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.453+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.453+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.453+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 206 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1a2eef70:::obj126:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.453+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a2eef70:::obj126:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 113 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.453+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a2eef70:::obj126:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e7b9c028:::obj510:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e7b9c028:::obj510:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 113/113 lb 1:e7b9c028:::obj510:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e7b9c028:::obj510:head) v3 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e74a4ee3:::obj396:head local-lis/les=109/110 n=550 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e7b9c028:::obj510:head) 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.453+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(4.0 113/112 [PushOp(4:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e7b9c028:::obj510:head) v3 finish latency 0.000015 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a2eef70:::obj126:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1a783fd9:::obj328:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1a783fd9:::obj328:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1a783fd9:::obj328:head 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1a783fd9:::obj328:head v 107'328 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.453+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.453+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1a783fd9:::obj328:head, version: 107'328, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1a783fd9:::obj328:head@107'328, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a2eef70:::obj126:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a2eef70:::obj126:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 113/113 lb 4:1a2eef70:::obj126:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1a2eef70:::obj126:head) v3 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a20990d:::obj103:head local-lis/les=112/113 n=79 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1a2eef70:::obj126:head) 2026-02-11T23:31:17.453+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a2eef70:::obj126:head) v3 finish latency 0.000011 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.453+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fcef00 con 0x3752000 2026-02-11T23:31:17.453+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e7f87738:::obj356:head)]) -- 0x497e480 con 0x3752000 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.453+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.453+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4fcf080 con 0x3e4a000 2026-02-11T23:31:17.453+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1a783fd9:::obj328:head)]) -- 0x49b8f00 con 0x3e4a000 2026-02-11T23:31:17.453+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1143 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2680 con 0x3752000 2026-02-11T23:31:17.454+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.454+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1144 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e7f87738:::obj356:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3752000 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.454+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e7f87738:::obj356:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 113 2026-02-11T23:31:17.454+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e7f87738:::obj356:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 113/109 [PushOp(1:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e7f87738:::obj356:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e8366861:::obj250:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e8366861:::obj250:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e8366861:::obj250:head 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e8366861:::obj250:head v 107'250 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.454+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e7f87738:::obj356:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e7f87738:::obj356:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 113/113 lb 1:e7f87738:::obj356:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e7f87738:::obj356:head) v3 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7b9c028:::obj510:head local-lis/les=109/110 n=551 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e7f87738:::obj356:head) 2026-02-11T23:31:17.454+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e7f87738:::obj356:head) v3 finish latency 0.000013 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 207 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3e4a000 2026-02-11T23:31:17.454+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.454+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 208 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1a783fd9:::obj328:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.454+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a783fd9:::obj328:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.454+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a783fd9:::obj328:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 113/112 [PushOp(4:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a783fd9:::obj328:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1abd4411:::obj66:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1abd4411:::obj66:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1abd4411:::obj66:head 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1abd4411:::obj66:head v 107'66 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.454+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1abd4411:::obj66:head, version: 107'66, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1abd4411:::obj66:head@107'66, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1a783fd9:::obj328:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a783fd9:::obj328:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 113/113 lb 4:1a783fd9:::obj328:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1a783fd9:::obj328:head) v3 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a2eef70:::obj126:head local-lis/les=112/113 n=80 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1a783fd9:::obj328:head) 2026-02-11T23:31:17.454+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1a783fd9:::obj328:head) v3 finish latency 0.000012 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.454+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fcf200 con 0x3752000 2026-02-11T23:31:17.454+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e8366861:::obj250:head)]) -- 0x4910600 con 0x3752000 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.454+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4fcf380 con 0x3e4a000 2026-02-11T23:31:17.454+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1abd4411:::obj66:head)]) -- 0x49b9080 con 0x3e4a000 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1145 ==== MOSDPGPush(1.0 113/109 [PushOp(1:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51824e0 con 0x3752000 2026-02-11T23:31:17.454+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.454+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1146 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e8366861:::obj250:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3752000 2026-02-11T23:31:17.454+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e8366861:::obj250:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.454+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e8366861:::obj250:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e8366861:::obj250:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:e9ac1484:::obj417:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:e9ac1484:::obj417:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:e9ac1484:::obj417:head 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:e9ac1484:::obj417:head v 107'417 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.454+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.454+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.454+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e8366861:::obj250:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e8366861:::obj250:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 1.0 e 113/113 lb 1:e8366861:::obj250:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e8366861:::obj250:head) v3 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e7f87738:::obj356:head local-lis/les=109/110 n=552 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e8366861:::obj250:head) 2026-02-11T23:31:17.454+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.454+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e8366861:::obj250:head) v3 finish latency 0.000013 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.454+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.455+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 209 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5182000 con 0x3e4a000 2026-02-11T23:31:17.455+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.455+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 210 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1abd4411:::obj66:head) ==== 1061+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.455+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1abd4411:::obj66:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.455+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1abd4411:::obj66:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1abd4411:::obj66:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1b644371:::obj48:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1b644371:::obj48:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1b644371:::obj48:head 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1b644371:::obj48:head v 107'48 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.455+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1b644371:::obj48:head, version: 107'48, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b644371:::obj48:head@107'48, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1abd4411:::obj66:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1abd4411:::obj66:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 113/113 lb 4:1abd4411:::obj66:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1abd4411:::obj66:head) v3 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1a783fd9:::obj328:head local-lis/les=112/113 n=81 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1abd4411:::obj66:head) 2026-02-11T23:31:17.455+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1abd4411:::obj66:head) v3 finish latency 0.000019 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.455+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fcf500 con 0x3752000 2026-02-11T23:31:17.455+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:e9ac1484:::obj417:head)]) -- 0x4dd7200 con 0x3752000 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.455+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4fcf680 con 0x3e4a000 2026-02-11T23:31:17.455+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1b644371:::obj48:head)]) -- 0x4dd6f00 con 0x3e4a000 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.455+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1147 ==== MOSDPGPush(1.0 113/109 [PushOp(1:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516da00 con 0x3752000 2026-02-11T23:31:17.455+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.455+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1148 ==== pg_backfill(progress 1.0 e 113/113 lb 1:e9ac1484:::obj417:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3752000 2026-02-11T23:31:17.455+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:e9ac1484:::obj417:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 113 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.455+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e9ac1484:::obj417:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 113/109 [PushOp(1:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e9ac1484:::obj417:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ea485012:::obj312:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ea485012:::obj312:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ea485012:::obj312:head 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ea485012:::obj312:head v 107'312 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.455+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:e9ac1484:::obj417:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e9ac1484:::obj417:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 1.0 e 113/113 lb 1:e9ac1484:::obj417:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:e9ac1484:::obj417:head) v3 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e8366861:::obj250:head local-lis/les=109/110 n=553 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:e9ac1484:::obj417:head) 2026-02-11T23:31:17.455+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:e9ac1484:::obj417:head) v3 finish latency 0.000019 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.455+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 211 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4de4680 con 0x3e4a000 2026-02-11T23:31:17.455+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.455+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 212 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1b644371:::obj48:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.455+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1b644371:::obj48:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.455+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1b644371:::obj48:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 113/112 [PushOp(4:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1b644371:::obj48:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1b6bffcc:::obj61:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1b6bffcc:::obj61:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1b6bffcc:::obj61:head 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1b6bffcc:::obj61:head v 107'61 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.455+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.455+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1b6bffcc:::obj61:head, version: 107'61, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6bffcc:::obj61:head@107'61, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1b644371:::obj48:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1b644371:::obj48:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 113/113 lb 4:1b644371:::obj48:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1b644371:::obj48:head) v3 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1abd4411:::obj66:head local-lis/les=112/113 n=82 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1b644371:::obj48:head) 2026-02-11T23:31:17.455+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1b644371:::obj48:head) v3 finish latency 0.000013 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.455+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fcf800 con 0x3752000 2026-02-11T23:31:17.455+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:ea485012:::obj312:head)]) -- 0x49b9380 con 0x3752000 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.455+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.455+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.455+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4fcf980 con 0x3e4a000 2026-02-11T23:31:17.455+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1b6bffcc:::obj61:head)]) -- 0x497e000 con 0x3e4a000 2026-02-11T23:31:17.455+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1149 ==== MOSDPGPush(1.0 113/109 [PushOp(1:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183040 con 0x3752000 2026-02-11T23:31:17.456+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.456+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1150 ==== pg_backfill(progress 1.0 e 113/113 lb 1:ea485012:::obj312:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3752000 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.456+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:ea485012:::obj312:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 113 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ea485012:::obj312:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 113/109 [PushOp(1:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ea485012:::obj312:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ea54a43e:::obj502:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ea54a43e:::obj502:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ea54a43e:::obj502:head 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ea54a43e:::obj502:head v 107'502 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.456+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ea485012:::obj312:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ea485012:::obj312:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 113/113 lb 1:ea485012:::obj312:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:ea485012:::obj312:head) v3 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:e9ac1484:::obj417:head local-lis/les=109/110 n=554 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:ea485012:::obj312:head) 2026-02-11T23:31:17.456+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ea485012:::obj312:head) v3 finish latency 0.000012 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 213 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.456+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.456+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 214 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1b6bffcc:::obj61:head) ==== 1061+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.456+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1b6bffcc:::obj61:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.456+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1b6bffcc:::obj61:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 113/112 [PushOp(4:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1b6bffcc:::obj61:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1b6ee6bf:::obj501:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1b6ee6bf:::obj501:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1b6ee6bf:::obj501:head 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1b6ee6bf:::obj501:head v 107'501 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.456+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1b6ee6bf:::obj501:head, version: 107'501, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1b6ee6bf:::obj501:head@107'501, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1b6bffcc:::obj61:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1b6bffcc:::obj61:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 113/113 lb 4:1b6bffcc:::obj61:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1b6bffcc:::obj61:head) v3 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b644371:::obj48:head local-lis/les=112/113 n=83 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1b6bffcc:::obj61:head) 2026-02-11T23:31:17.456+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1b6bffcc:::obj61:head) v3 finish latency 0.000013 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.456+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fcfb00 con 0x3752000 2026-02-11T23:31:17.456+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:ea54a43e:::obj502:head)]) -- 0x4dd6a80 con 0x3752000 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.456+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4fcfc80 con 0x3e4a000 2026-02-11T23:31:17.456+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1b6ee6bf:::obj501:head)]) -- 0x4dd6780 con 0x3e4a000 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1151 ==== MOSDPGPush(1.0 113/109 [PushOp(1:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3752000 2026-02-11T23:31:17.456+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.456+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.456+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1152 ==== pg_backfill(progress 1.0 e 113/113 lb 1:ea54a43e:::obj502:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3752000 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:ea54a43e:::obj502:head) v3 prio 15 type 95 cost 0 latency 0.000014 epoch 113 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.456+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ea54a43e:::obj502:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 113/109 [PushOp(1:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ea54a43e:::obj502:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:eaf65c1c:::obj380:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:eaf65c1c:::obj380:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:eaf65c1c:::obj380:head 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:eaf65c1c:::obj380:head v 107'380 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.456+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.456+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ea54a43e:::obj502:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ea54a43e:::obj502:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 113/113 lb 1:ea54a43e:::obj502:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:ea54a43e:::obj502:head) v3 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea485012:::obj312:head local-lis/les=109/110 n=555 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:ea54a43e:::obj502:head) 2026-02-11T23:31:17.456+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ea54a43e:::obj502:head) v3 finish latency 0.000014 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.456+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.456+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fcfe00 con 0x3752000 2026-02-11T23:31:17.456+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:eaf65c1c:::obj380:head)]) -- 0x4dd6480 con 0x3752000 2026-02-11T23:31:17.456+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.457+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 215 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.457+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 113 2026-02-11T23:31:17.457+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 216 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1b6ee6bf:::obj501:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3e4a000 2026-02-11T23:31:17.457+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1b6ee6bf:::obj501:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.457+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1b6ee6bf:::obj501:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 113/112 [PushOp(4:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1b6ee6bf:::obj501:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1bad8e21:::obj122:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1bad8e21:::obj122:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1bad8e21:::obj122:head 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1bad8e21:::obj122:head v 107'122 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.457+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1bad8e21:::obj122:head, version: 107'122, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bad8e21:::obj122:head@107'122, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1b6ee6bf:::obj501:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1b6ee6bf:::obj501:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 113/113 lb 4:1b6ee6bf:::obj501:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1b6ee6bf:::obj501:head) v3 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6bffcc:::obj61:head local-lis/les=112/113 n=84 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1b6ee6bf:::obj501:head) 2026-02-11T23:31:17.457+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1b6ee6bf:::obj501:head) v3 finish latency 0.000010 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.457+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1153 ==== MOSDPGPush(1.0 113/109 [PushOp(1:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3752000 2026-02-11T23:31:17.457+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.457+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1154 ==== pg_backfill(progress 1.0 e 113/113 lb 1:eaf65c1c:::obj380:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3752000 2026-02-11T23:31:17.457+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:eaf65c1c:::obj380:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.457+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eaf65c1c:::obj380:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eaf65c1c:::obj380:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:eb0c167c:::obj188:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:eb0c167c:::obj188:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:eb0c167c:::obj188:head 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:eb0c167c:::obj188:head v 107'188 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.457+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eaf65c1c:::obj380:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eaf65c1c:::obj380:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 1.0 e 113/113 lb 1:eaf65c1c:::obj380:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:eaf65c1c:::obj380:head) v3 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ea54a43e:::obj502:head local-lis/les=109/110 n=556 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:eaf65c1c:::obj380:head) 2026-02-11T23:31:17.457+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eaf65c1c:::obj380:head) v3 finish latency 0.000010 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.457+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96000 con 0x3e4a000 2026-02-11T23:31:17.457+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1bad8e21:::obj122:head)]) -- 0x49d6600 con 0x3e4a000 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.457+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c96180 con 0x3752000 2026-02-11T23:31:17.457+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:eb0c167c:::obj188:head)]) -- 0x4910000 con 0x3752000 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.457+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1155 ==== MOSDPGPush(1.0 113/109 [PushOp(1:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3752000 2026-02-11T23:31:17.457+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.457+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1156 ==== pg_backfill(progress 1.0 e 113/113 lb 1:eb0c167c:::obj188:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3752000 2026-02-11T23:31:17.457+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:eb0c167c:::obj188:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 113 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.457+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eb0c167c:::obj188:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 113/109 [PushOp(1:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eb0c167c:::obj188:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.457+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 217 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3e4a000 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:eb29d295:::obj554:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:eb29d295:::obj554:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.457+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:eb29d295:::obj554:head 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:eb29d295:::obj554:head v 107'554 2026-02-11T23:31:17.457+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.457+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.457+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 218 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1bad8e21:::obj122:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3e4a000 2026-02-11T23:31:17.457+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1bad8e21:::obj122:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.457+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1bad8e21:::obj122:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000055 MOSDPGPush(4.0 113/112 [PushOp(4:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eb0c167c:::obj188:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eb0c167c:::obj188:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 113/113 lb 1:eb0c167c:::obj188:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:eb0c167c:::obj188:head) v3 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eaf65c1c:::obj380:head local-lis/les=109/110 n=557 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:eb0c167c:::obj188:head) 2026-02-11T23:31:17.457+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1bb1e69e:::obj390:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1bb1e69e:::obj390:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1bb1e69e:::obj390:head 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eb0c167c:::obj188:head) v3 finish latency 0.000024 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1bb1e69e:::obj390:head v 107'390 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1bad8e21:::obj122:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.457+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.457+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1bb1e69e:::obj390:head, version: 107'390, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1bb1e69e:::obj390:head@107'390, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.457+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1bad8e21:::obj122:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1bad8e21:::obj122:head) v3 prio 15 cost 0 latency 0.000108 pg_backfill(progress 4.0 e 113/113 lb 4:1bad8e21:::obj122:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1bad8e21:::obj122:head) v3 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1b6ee6bf:::obj501:head local-lis/les=112/113 n=85 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1bad8e21:::obj122:head) 2026-02-11T23:31:17.457+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.457+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1bad8e21:::obj122:head) v3 finish latency 0.000009 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.457+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.458+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c96300 con 0x3752000 2026-02-11T23:31:17.458+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:eb29d295:::obj554:head)]) -- 0x49b8780 con 0x3752000 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.458+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96480 con 0x3e4a000 2026-02-11T23:31:17.458+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1bb1e69e:::obj390:head)]) -- 0x4910180 con 0x3e4a000 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1157 ==== MOSDPGPush(1.0 113/109 [PushOp(1:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3752000 2026-02-11T23:31:17.458+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.458+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1158 ==== pg_backfill(progress 1.0 e 113/113 lb 1:eb29d295:::obj554:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3752000 2026-02-11T23:31:17.458+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:eb29d295:::obj554:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.458+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eb29d295:::obj554:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 113/109 [PushOp(1:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eb29d295:::obj554:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:eb806c57:::obj373:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:eb806c57:::obj373:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:eb806c57:::obj373:head 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:eb806c57:::obj373:head v 107'373 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.458+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eb29d295:::obj554:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eb29d295:::obj554:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 113/113 lb 1:eb29d295:::obj554:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:eb29d295:::obj554:head) v3 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb0c167c:::obj188:head local-lis/les=109/110 n=558 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:eb29d295:::obj554:head) 2026-02-11T23:31:17.458+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eb29d295:::obj554:head) v3 finish latency 0.000025 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.458+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c96600 con 0x3752000 2026-02-11T23:31:17.458+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:eb806c57:::obj373:head)]) -- 0x4a04900 con 0x3752000 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 219 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3e4a000 2026-02-11T23:31:17.458+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.458+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 220 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1bb1e69e:::obj390:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3e4a000 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.458+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1bb1e69e:::obj390:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1bb1e69e:::obj390:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 113/112 [PushOp(4:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1bb1e69e:::obj390:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1c47bf8e:::obj558:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1c47bf8e:::obj558:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1c47bf8e:::obj558:head 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1c47bf8e:::obj558:head v 107'558 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.458+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1c47bf8e:::obj558:head, version: 107'558, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1c47bf8e:::obj558:head@107'558, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1bb1e69e:::obj390:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1bb1e69e:::obj390:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 113/113 lb 4:1bb1e69e:::obj390:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1bb1e69e:::obj390:head) v3 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bad8e21:::obj122:head local-lis/les=112/113 n=86 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1bb1e69e:::obj390:head) 2026-02-11T23:31:17.458+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1bb1e69e:::obj390:head) v3 finish latency 0.000014 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1159 ==== MOSDPGPush(1.0 113/109 [PushOp(1:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3752000 2026-02-11T23:31:17.458+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.458+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1160 ==== pg_backfill(progress 1.0 e 113/113 lb 1:eb806c57:::obj373:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3752000 2026-02-11T23:31:17.458+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:eb806c57:::obj373:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 113 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.458+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eb806c57:::obj373:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 113/109 [PushOp(1:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eb806c57:::obj373:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ecbe4ab7:::obj125:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ecbe4ab7:::obj125:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ecbe4ab7:::obj125:head 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ecbe4ab7:::obj125:head v 107'125 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.458+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.458+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.458+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eb806c57:::obj373:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eb806c57:::obj373:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 113/113 lb 1:eb806c57:::obj373:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:eb806c57:::obj373:head) v3 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb29d295:::obj554:head local-lis/les=109/110 n=559 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:eb806c57:::obj373:head) 2026-02-11T23:31:17.458+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eb806c57:::obj373:head) v3 finish latency 0.000011 2026-02-11T23:31:17.458+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.458+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96780 con 0x3e4a000 2026-02-11T23:31:17.458+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1c47bf8e:::obj558:head)]) -- 0x49d6f00 con 0x3e4a000 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.458+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.459+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c96900 con 0x3752000 2026-02-11T23:31:17.459+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:ecbe4ab7:::obj125:head)]) -- 0x4916000 con 0x3752000 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.459+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 221 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3e4a000 2026-02-11T23:31:17.459+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.459+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.459+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 222 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1c47bf8e:::obj558:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3e4a000 2026-02-11T23:31:17.459+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1c47bf8e:::obj558:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 113 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.459+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1c47bf8e:::obj558:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 113/112 [PushOp(4:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1c47bf8e:::obj558:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1cb33d95:::obj190:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1cb33d95:::obj190:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1cb33d95:::obj190:head 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1cb33d95:::obj190:head v 107'190 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.459+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.459+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1161 ==== MOSDPGPush(1.0 113/109 [PushOp(1:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3752000 2026-02-11T23:31:17.459+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.459+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.459+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1162 ==== pg_backfill(progress 1.0 e 113/113 lb 1:ecbe4ab7:::obj125:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3752000 2026-02-11T23:31:17.459+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:ecbe4ab7:::obj125:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.459+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ecbe4ab7:::obj125:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cb33d95:::obj190:head, version: 107'190, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cb33d95:::obj190:head@107'190, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(1.0 113/109 [PushOp(1:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1c47bf8e:::obj558:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1c47bf8e:::obj558:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 4.0 e 113/113 lb 4:1c47bf8e:::obj558:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1c47bf8e:::obj558:head) v3 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1bb1e69e:::obj390:head local-lis/les=112/113 n=87 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1c47bf8e:::obj558:head) 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ed234181:::obj472:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.459+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ed234181:::obj472:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ed234181:::obj472:head 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ed234181:::obj472:head v 107'472 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1c47bf8e:::obj558:head) v3 finish latency 0.000012 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ecbe4ab7:::obj125:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.459+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.459+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ecbe4ab7:::obj125:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ecbe4ab7:::obj125:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 1.0 e 113/113 lb 1:ecbe4ab7:::obj125:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:ecbe4ab7:::obj125:head) v3 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eb806c57:::obj373:head local-lis/les=109/110 n=560 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:ecbe4ab7:::obj125:head) 2026-02-11T23:31:17.459+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ecbe4ab7:::obj125:head) v3 finish latency 0.000010 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.459+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96a80 con 0x3e4a000 2026-02-11T23:31:17.459+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1cb33d95:::obj190:head)]) -- 0x49d6c00 con 0x3e4a000 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.459+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.459+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.459+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c96c00 con 0x3752000 2026-02-11T23:31:17.459+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:ed234181:::obj472:head)]) -- 0x49d7080 con 0x3752000 2026-02-11T23:31:17.459+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 223 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3e4a000 2026-02-11T23:31:17.460+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.460+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 224 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1cb33d95:::obj190:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3e4a000 2026-02-11T23:31:17.460+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cb33d95:::obj190:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.460+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cb33d95:::obj190:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cb33d95:::obj190:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 113/112 [PushOp(4:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.460+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1163 ==== MOSDPGPush(1.0 113/109 [PushOp(1:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3752000 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1cd4c3d5:::obj142:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.460+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1cd4c3d5:::obj142:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1cd4c3d5:::obj142:head 2026-02-11T23:31:17.460+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1cd4c3d5:::obj142:head v 107'142 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.460+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.460+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1164 ==== pg_backfill(progress 1.0 e 113/113 lb 1:ed234181:::obj472:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d700 con 0x3752000 2026-02-11T23:31:17.460+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:ed234181:::obj472:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.460+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ed234181:::obj472:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cd4c3d5:::obj142:head, version: 107'142, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cd4c3d5:::obj142:head@107'142, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000058 MOSDPGPush(1.0 113/109 [PushOp(1:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cb33d95:::obj190:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cb33d95:::obj190:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 4.0 e 113/113 lb 4:1cb33d95:::obj190:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1cb33d95:::obj190:head) v3 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1c47bf8e:::obj558:head local-lis/les=112/113 n=88 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1cb33d95:::obj190:head) 2026-02-11T23:31:17.460+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ee44f25c:::obj334:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cb33d95:::obj190:head) v3 finish latency 0.000018 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ee44f25c:::obj334:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ed234181:::obj472:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ee44f25c:::obj334:head 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ee44f25c:::obj334:head v 107'334 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.460+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ed234181:::obj472:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ed234181:::obj472:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 1.0 e 113/113 lb 1:ed234181:::obj472:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:ed234181:::obj472:head) v3 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ecbe4ab7:::obj125:head local-lis/les=109/110 n=561 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:ed234181:::obj472:head) 2026-02-11T23:31:17.460+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ed234181:::obj472:head) v3 finish latency 0.000010 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.460+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96d80 con 0x3e4a000 2026-02-11T23:31:17.460+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1cd4c3d5:::obj142:head)]) -- 0x49d7380 con 0x3e4a000 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.460+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c96f00 con 0x3752000 2026-02-11T23:31:17.460+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:ee44f25c:::obj334:head)]) -- 0x49d7200 con 0x3752000 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 225 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d6c0 con 0x3e4a000 2026-02-11T23:31:17.460+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.460+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.460+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 226 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1cd4c3d5:::obj142:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3e4a000 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cd4c3d5:::obj142:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.460+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cd4c3d5:::obj142:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 113/112 [PushOp(4:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cd4c3d5:::obj142:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1cf1ad59:::obj156:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1cf1ad59:::obj156:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1cf1ad59:::obj156:head 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1cf1ad59:::obj156:head v 107'156 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.460+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cf1ad59:::obj156:head, version: 107'156, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cf1ad59:::obj156:head@107'156, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cd4c3d5:::obj142:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cd4c3d5:::obj142:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 113/113 lb 4:1cd4c3d5:::obj142:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1cd4c3d5:::obj142:head) v3 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cb33d95:::obj190:head local-lis/les=112/113 n=89 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1cd4c3d5:::obj142:head) 2026-02-11T23:31:17.460+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cd4c3d5:::obj142:head) v3 finish latency 0.000011 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1165 ==== MOSDPGPush(1.0 113/109 [PushOp(1:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x389b040 con 0x3752000 2026-02-11T23:31:17.460+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.460+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.460+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1166 ==== pg_backfill(progress 1.0 e 113/113 lb 1:ee44f25c:::obj334:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3752000 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:ee44f25c:::obj334:head) v3 prio 15 type 95 cost 0 latency 0.000012 epoch 113 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.460+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ee44f25c:::obj334:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ee44f25c:::obj334:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:eeae2a94:::obj11:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:eeae2a94:::obj11:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:eeae2a94:::obj11:head 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:eeae2a94:::obj11:head v 107'11 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.460+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.460+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000039 2026-02-11T23:31:17.460+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ee44f25c:::obj334:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ee44f25c:::obj334:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 1.0 e 113/113 lb 1:ee44f25c:::obj334:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:ee44f25c:::obj334:head) v3 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ed234181:::obj472:head local-lis/les=109/110 n=562 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:ee44f25c:::obj334:head) 2026-02-11T23:31:17.460+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.460+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ee44f25c:::obj334:head) v3 finish latency 0.000010 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.460+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.461+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.461+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1cf1ad59:::obj156:head)]) -- 0x49d7800 con 0x3e4a000 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.461+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c97200 con 0x3752000 2026-02-11T23:31:17.461+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:eeae2a94:::obj11:head)]) -- 0x49d7500 con 0x3752000 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1167 ==== MOSDPGPush(1.0 113/109 [PushOp(1:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d040 con 0x3752000 2026-02-11T23:31:17.461+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.461+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.461+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1168 ==== pg_backfill(progress 1.0 e 113/113 lb 1:eeae2a94:::obj11:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3752000 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.461+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:eeae2a94:::obj11:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.461+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eeae2a94:::obj11:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(1.0 113/109 [PushOp(1:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eeae2a94:::obj11:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:eff0a739:::obj127:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:eff0a739:::obj127:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:eff0a739:::obj127:head 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:eff0a739:::obj127:head v 107'127 2026-02-11T23:31:17.461+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 227 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de44e0 con 0x3e4a000 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.461+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.461+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.461+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.461+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 228 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1cf1ad59:::obj156:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3e4a000 2026-02-11T23:31:17.461+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cf1ad59:::obj156:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.461+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cf1ad59:::obj156:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eeae2a94:::obj11:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(4.0 113/112 [PushOp(4:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eeae2a94:::obj11:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 1.0 e 113/113 lb 1:eeae2a94:::obj11:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:eeae2a94:::obj11:head) v3 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ee44f25c:::obj334:head local-lis/les=109/110 n=563 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:eeae2a94:::obj11:head) 2026-02-11T23:31:17.461+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eeae2a94:::obj11:head) v3 finish latency 0.000011 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1cff0951:::obj258:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1cff0951:::obj258:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cf1ad59:::obj156:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1cff0951:::obj258:head 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1cff0951:::obj258:head v 107'258 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.461+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.461+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1cff0951:::obj258:head, version: 107'258, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1cff0951:::obj258:head@107'258, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cf1ad59:::obj156:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cf1ad59:::obj156:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 113/113 lb 4:1cf1ad59:::obj156:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1cf1ad59:::obj156:head) v3 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cd4c3d5:::obj142:head local-lis/les=112/113 n=90 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1cf1ad59:::obj156:head) 2026-02-11T23:31:17.461+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cf1ad59:::obj156:head) v3 finish latency 0.000010 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.461+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c97380 con 0x3752000 2026-02-11T23:31:17.461+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:eff0a739:::obj127:head)]) -- 0x4911c80 con 0x3752000 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.461+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c97500 con 0x3e4a000 2026-02-11T23:31:17.461+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1cff0951:::obj258:head)]) -- 0x4a04c00 con 0x3e4a000 2026-02-11T23:31:17.461+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.461+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1169 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183380 con 0x3752000 2026-02-11T23:31:17.462+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 113 2026-02-11T23:31:17.462+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.462+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1170 ==== pg_backfill(progress 1.0 e 113/113 lb 1:eff0a739:::obj127:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162000 con 0x3752000 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:eff0a739:::obj127:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.462+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eff0a739:::obj127:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eff0a739:::obj127:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f0a12295:::obj487:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f0a12295:::obj487:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f0a12295:::obj487:head 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f0a12295:::obj487:head v 107'487 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.462+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:eff0a739:::obj127:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eff0a739:::obj127:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 113/113 lb 1:eff0a739:::obj127:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:eff0a739:::obj127:head) v3 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eeae2a94:::obj11:head local-lis/les=109/110 n=564 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:eff0a739:::obj127:head) 2026-02-11T23:31:17.462+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:eff0a739:::obj127:head) v3 finish latency 0.000012 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.462+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c97680 con 0x3752000 2026-02-11T23:31:17.462+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f0a12295:::obj487:head)]) -- 0x4eabc80 con 0x3752000 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 229 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dd40 con 0x3e4a000 2026-02-11T23:31:17.462+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.462+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 230 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1cff0951:::obj258:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162580 con 0x3e4a000 2026-02-11T23:31:17.462+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cff0951:::obj258:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.462+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cff0951:::obj258:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 113/112 [PushOp(4:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cff0951:::obj258:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1d2e941a:::obj186:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1d2e941a:::obj186:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1d2e941a:::obj186:head 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1d2e941a:::obj186:head v 107'186 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.462+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1d2e941a:::obj186:head, version: 107'186, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1d2e941a:::obj186:head@107'186, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1cff0951:::obj258:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cff0951:::obj258:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 113/113 lb 4:1cff0951:::obj258:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1cff0951:::obj258:head) v3 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cf1ad59:::obj156:head local-lis/les=112/113 n=91 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1cff0951:::obj258:head) 2026-02-11T23:31:17.462+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1cff0951:::obj258:head) v3 finish latency 0.000011 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1171 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3752000 2026-02-11T23:31:17.462+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.462+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.462+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c97800 con 0x3e4a000 2026-02-11T23:31:17.462+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1d2e941a:::obj186:head)]) -- 0x4c96180 con 0x3e4a000 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 113/109 [PushOp(1:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.462+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1172 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f0a12295:::obj487:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3752000 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.462+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f0a12295:::obj487:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.462+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f0a12295:::obj487:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f1182822:::obj193:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f1182822:::obj193:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f0a12295:::obj487:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f1182822:::obj193:head 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f1182822:::obj193:head v 107'193 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.462+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f0a12295:::obj487:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f0a12295:::obj487:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 1.0 e 113/113 lb 1:f0a12295:::obj487:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f0a12295:::obj487:head) v3 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:eff0a739:::obj127:head local-lis/les=109/110 n=565 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f0a12295:::obj487:head) 2026-02-11T23:31:17.462+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f0a12295:::obj487:head) v3 finish latency 0.000012 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.462+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c97980 con 0x3752000 2026-02-11T23:31:17.462+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f1182822:::obj193:head)]) -- 0x4c96000 con 0x3752000 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.462+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 231 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43ea1a0 con 0x3e4a000 2026-02-11T23:31:17.462+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.462+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 232 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1d2e941a:::obj186:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163080 con 0x3e4a000 2026-02-11T23:31:17.462+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1d2e941a:::obj186:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.462+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1d2e941a:::obj186:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 113/112 [PushOp(4:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.462+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1d2e941a:::obj186:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1dafe4f0:::obj47:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.462+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1dafe4f0:::obj47:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1dafe4f0:::obj47:head 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1dafe4f0:::obj47:head v 107'47 2026-02-11T23:31:17.462+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.462+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1dafe4f0:::obj47:head, version: 107'47, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1dafe4f0:::obj47:head@107'47, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1d2e941a:::obj186:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1d2e941a:::obj186:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 113/113 lb 4:1d2e941a:::obj186:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1d2e941a:::obj186:head) v3 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1cff0951:::obj258:head local-lis/les=112/113 n=92 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1d2e941a:::obj186:head) 2026-02-11T23:31:17.463+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1d2e941a:::obj186:head) v3 finish latency 0.000011 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.463+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c97b00 con 0x3e4a000 2026-02-11T23:31:17.463+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1dafe4f0:::obj47:head)]) -- 0x4fcfe00 con 0x3e4a000 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1173 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3752000 2026-02-11T23:31:17.463+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.463+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1174 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f1182822:::obj193:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163600 con 0x3752000 2026-02-11T23:31:17.463+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f1182822:::obj193:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.463+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f1182822:::obj193:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f1182822:::obj193:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f11aef48:::obj58:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f11aef48:::obj58:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f11aef48:::obj58:head 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f11aef48:::obj58:head v 107'58 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.463+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f1182822:::obj193:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f1182822:::obj193:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 113/113 lb 1:f1182822:::obj193:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f1182822:::obj193:head) v3 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f0a12295:::obj487:head local-lis/les=109/110 n=566 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f1182822:::obj193:head) 2026-02-11T23:31:17.463+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f1182822:::obj193:head) v3 finish latency 0.000011 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.463+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c97c80 con 0x3752000 2026-02-11T23:31:17.463+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f11aef48:::obj58:head)]) -- 0x4dd6780 con 0x3752000 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 233 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.463+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.463+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 234 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1dafe4f0:::obj47:head) ==== 1061+0+0 (unknown 0 0 0) 0x5163b80 con 0x3e4a000 2026-02-11T23:31:17.463+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1dafe4f0:::obj47:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.463+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1dafe4f0:::obj47:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1dafe4f0:::obj47:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 113/112 [PushOp(4:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1e53d0d0:::obj151:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1e53d0d0:::obj151:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1e53d0d0:::obj151:head 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1e53d0d0:::obj151:head v 107'151 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.463+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1e53d0d0:::obj151:head, version: 107'151, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1e53d0d0:::obj151:head@107'151, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1dafe4f0:::obj47:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1dafe4f0:::obj47:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 113/113 lb 4:1dafe4f0:::obj47:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1dafe4f0:::obj47:head) v3 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1d2e941a:::obj186:head local-lis/les=112/113 n=93 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1dafe4f0:::obj47:head) 2026-02-11T23:31:17.463+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1dafe4f0:::obj47:head) v3 finish latency 0.000011 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.463+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c97e00 con 0x3e4a000 2026-02-11T23:31:17.463+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1e53d0d0:::obj151:head)]) -- 0x4fcfc80 con 0x3e4a000 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1175 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3752000 2026-02-11T23:31:17.463+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.463+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1176 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f11aef48:::obj58:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164680 con 0x3752000 2026-02-11T23:31:17.463+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f11aef48:::obj58:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.463+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f11aef48:::obj58:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(1.0 113/109 [PushOp(1:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f11aef48:::obj58:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f13c731e:::obj449:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f13c731e:::obj449:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f13c731e:::obj449:head 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f13c731e:::obj449:head v 107'449 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.463+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.463+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.463+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f11aef48:::obj58:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f11aef48:::obj58:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 113/113 lb 1:f11aef48:::obj58:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f11aef48:::obj58:head) v3 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f1182822:::obj193:head local-lis/les=109/110 n=567 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f11aef48:::obj58:head) 2026-02-11T23:31:17.463+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.463+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f11aef48:::obj58:head) v3 finish latency 0.000011 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.463+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.464+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c97980 con 0x3752000 2026-02-11T23:31:17.464+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f13c731e:::obj449:head)]) -- 0x4fcfb00 con 0x3752000 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.464+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 235 ==== MOSDPGPush(4.0 113/112 [PushOp(4:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.464+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.464+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.464+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 236 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1e53d0d0:::obj151:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.464+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1e53d0d0:::obj151:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.464+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1e53d0d0:::obj151:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1e53d0d0:::obj151:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:1efaad2b:::obj120:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:1efaad2b:::obj120:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:1efaad2b:::obj120:head 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:1efaad2b:::obj120:head v 107'120 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.464+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:1efaad2b:::obj120:head, version: 107'120, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:1efaad2b:::obj120:head@107'120, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1e53d0d0:::obj151:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1e53d0d0:::obj151:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 113/113 lb 4:1e53d0d0:::obj151:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1e53d0d0:::obj151:head) v3 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1dafe4f0:::obj47:head local-lis/les=112/113 n=94 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1e53d0d0:::obj151:head) 2026-02-11T23:31:17.464+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1e53d0d0:::obj151:head) v3 finish latency 0.000011 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.464+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1177 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3752000 2026-02-11T23:31:17.464+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.464+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.464+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1178 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f13c731e:::obj449:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3752000 2026-02-11T23:31:17.464+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f13c731e:::obj449:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.464+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f13c731e:::obj449:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f13c731e:::obj449:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f22dcb20:::obj569:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f22dcb20:::obj569:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f22dcb20:::obj569:head 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f22dcb20:::obj569:head v 107'569 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.464+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.464+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f13c731e:::obj449:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f13c731e:::obj449:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 1.0 e 113/113 lb 1:f13c731e:::obj449:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f13c731e:::obj449:head) v3 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f11aef48:::obj58:head local-lis/les=109/110 n=568 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f13c731e:::obj449:head) 2026-02-11T23:31:17.464+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f13c731e:::obj449:head) v3 finish latency 0.000010 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.464+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96180 con 0x3e4a000 2026-02-11T23:31:17.464+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:1efaad2b:::obj120:head)]) -- 0x4fcf980 con 0x3e4a000 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.464+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.464+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c97800 con 0x3752000 2026-02-11T23:31:17.464+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f22dcb20:::obj569:head)]) -- 0x4fcf800 con 0x3752000 2026-02-11T23:31:17.464+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.464+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 237 ==== MOSDPGPush(4.0 113/112 [PushOp(4:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a21a0 con 0x3e4a000 2026-02-11T23:31:17.464+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.464+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.464+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 238 ==== pg_backfill(progress 4.0 e 113/113 lb 4:1efaad2b:::obj120:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.464+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.464+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:1efaad2b:::obj120:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1efaad2b:::obj120:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 113/112 [PushOp(4:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1efaad2b:::obj120:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:20b87b8b:::obj369:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:20b87b8b:::obj369:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:20b87b8b:::obj369:head 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:20b87b8b:::obj369:head v 107'369 2026-02-11T23:31:17.465+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1179 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2680 con 0x3752000 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.465+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.465+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 113 2026-02-11T23:31:17.465+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.465+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1180 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f22dcb20:::obj569:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3752000 2026-02-11T23:31:17.465+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f22dcb20:::obj569:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.465+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f22dcb20:::obj569:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:20b87b8b:::obj369:head, version: 107'369, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:20b87b8b:::obj369:head@107'369, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000057 MOSDPGPush(1.0 113/109 [PushOp(1:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:1efaad2b:::obj120:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1efaad2b:::obj120:head) v3 prio 15 cost 0 latency 0.000099 pg_backfill(progress 4.0 e 113/113 lb 4:1efaad2b:::obj120:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:1efaad2b:::obj120:head) v3 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f2a1ba99:::obj364:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1e53d0d0:::obj151:head local-lis/les=112/113 n=95 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:1efaad2b:::obj120:head) 2026-02-11T23:31:17.465+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:1efaad2b:::obj120:head) v3 finish latency 0.000013 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f2a1ba99:::obj364:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f2a1ba99:::obj364:head 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f22dcb20:::obj569:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f2a1ba99:::obj364:head v 107'364 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.465+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f22dcb20:::obj569:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f22dcb20:::obj569:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 1.0 e 113/113 lb 1:f22dcb20:::obj569:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f22dcb20:::obj569:head) v3 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f13c731e:::obj449:head local-lis/les=109/110 n=569 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f22dcb20:::obj569:head) 2026-02-11T23:31:17.465+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f22dcb20:::obj569:head) v3 finish latency 0.000010 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.465+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4eabc80 con 0x3e4a000 2026-02-11T23:31:17.465+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:20b87b8b:::obj369:head)]) -- 0x4fcf680 con 0x3e4a000 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.465+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c97680 con 0x3752000 2026-02-11T23:31:17.465+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f2a1ba99:::obj364:head)]) -- 0x4fcf500 con 0x3752000 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.465+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 239 ==== MOSDPGPush(4.0 113/112 [PushOp(4:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2b60 con 0x3e4a000 2026-02-11T23:31:17.465+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 113 2026-02-11T23:31:17.465+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 240 ==== pg_backfill(progress 4.0 e 113/113 lb 4:20b87b8b:::obj369:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:20b87b8b:::obj369:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 113 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.465+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:20b87b8b:::obj369:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 113/112 [PushOp(4:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:20b87b8b:::obj369:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:212e6483:::obj209:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:212e6483:::obj209:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:212e6483:::obj209:head 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:212e6483:::obj209:head v 107'209 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.465+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:212e6483:::obj209:head, version: 107'209, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:212e6483:::obj209:head@107'209, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:20b87b8b:::obj369:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:20b87b8b:::obj369:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 113/113 lb 4:20b87b8b:::obj369:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:20b87b8b:::obj369:head) v3 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:1efaad2b:::obj120:head local-lis/les=112/113 n=96 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:20b87b8b:::obj369:head) 2026-02-11T23:31:17.465+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:20b87b8b:::obj369:head) v3 finish latency 0.000011 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.465+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1181 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4dfe1a0 con 0x3752000 2026-02-11T23:31:17.465+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.465+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1182 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f2a1ba99:::obj364:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3752000 2026-02-11T23:31:17.465+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2a1ba99:::obj364:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.465+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2a1ba99:::obj364:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2a1ba99:::obj364:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f2bc7f0e:::obj94:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f2bc7f0e:::obj94:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f2bc7f0e:::obj94:head 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f2bc7f0e:::obj94:head v 107'94 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.465+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.465+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.465+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2a1ba99:::obj364:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2a1ba99:::obj364:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 113/113 lb 1:f2a1ba99:::obj364:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f2a1ba99:::obj364:head) v3 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f22dcb20:::obj569:head local-lis/les=109/110 n=570 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f2a1ba99:::obj364:head) 2026-02-11T23:31:17.465+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2a1ba99:::obj364:head) v3 finish latency 0.000011 2026-02-11T23:31:17.465+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.465+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4a04c00 con 0x3e4a000 2026-02-11T23:31:17.465+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:212e6483:::obj209:head)]) -- 0x4fcf380 con 0x3e4a000 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.465+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.466+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c97500 con 0x3752000 2026-02-11T23:31:17.466+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f2bc7f0e:::obj94:head)]) -- 0x4fcf200 con 0x3752000 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 241 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3520 con 0x3e4a000 2026-02-11T23:31:17.466+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.466+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.466+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 242 ==== pg_backfill(progress 4.0 e 113/113 lb 4:212e6483:::obj209:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3e4a000 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.466+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:212e6483:::obj209:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 113 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.466+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:212e6483:::obj209:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 113/112 [PushOp(4:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:212e6483:::obj209:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2222704f:::obj161:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2222704f:::obj161:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2222704f:::obj161:head 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2222704f:::obj161:head v 107'161 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.466+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2222704f:::obj161:head, version: 107'161, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2222704f:::obj161:head@107'161, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:212e6483:::obj209:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:212e6483:::obj209:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 113/113 lb 4:212e6483:::obj209:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.466+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1183 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183d40 con 0x3752000 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:212e6483:::obj209:head) v3 2026-02-11T23:31:17.466+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:20b87b8b:::obj369:head local-lis/les=112/113 n=97 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:212e6483:::obj209:head) 2026-02-11T23:31:17.466+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.466+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:212e6483:::obj209:head) v3 finish latency 0.000012 2026-02-11T23:31:17.466+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1184 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f2bc7f0e:::obj94:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5600 con 0x3752000 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.466+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2bc7f0e:::obj94:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.466+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2bc7f0e:::obj94:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2bc7f0e:::obj94:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f2f9dd5a:::obj121:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f2f9dd5a:::obj121:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f2f9dd5a:::obj121:head 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f2f9dd5a:::obj121:head v 107'121 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.466+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2bc7f0e:::obj94:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2bc7f0e:::obj94:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 113/113 lb 1:f2bc7f0e:::obj94:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f2bc7f0e:::obj94:head) v3 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2a1ba99:::obj364:head local-lis/les=109/110 n=571 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f2bc7f0e:::obj94:head) 2026-02-11T23:31:17.466+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2bc7f0e:::obj94:head) v3 finish latency 0.000011 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.466+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49d7500 con 0x3e4a000 2026-02-11T23:31:17.466+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2222704f:::obj161:head)]) -- 0x49b8f00 con 0x3e4a000 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.466+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c97200 con 0x3752000 2026-02-11T23:31:17.466+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f2f9dd5a:::obj121:head)]) -- 0x4911c80 con 0x3752000 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1185 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3752000 2026-02-11T23:31:17.466+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.466+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.466+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1186 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f2f9dd5a:::obj121:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3752000 2026-02-11T23:31:17.466+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.466+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2f9dd5a:::obj121:head) v3 prio 15 type 95 cost 0 latency 0.000014 epoch 113 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.466+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2f9dd5a:::obj121:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.466+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.466+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2f9dd5a:::obj121:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f2fc2104:::obj429:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f2fc2104:::obj429:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f2fc2104:::obj429:head 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f2fc2104:::obj429:head v 107'429 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.467+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 243 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.467+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.467+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.467+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.467+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 244 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2222704f:::obj161:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3e4a000 2026-02-11T23:31:17.467+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2222704f:::obj161:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.467+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2222704f:::obj161:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000047 MOSDPGPush(4.0 113/112 [PushOp(4:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2f9dd5a:::obj121:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2f9dd5a:::obj121:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 113/113 lb 1:f2f9dd5a:::obj121:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f2f9dd5a:::obj121:head) v3 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2bc7f0e:::obj94:head local-lis/les=109/110 n=572 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f2f9dd5a:::obj121:head) 2026-02-11T23:31:17.467+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2260a46a:::obj167:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2260a46a:::obj167:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2260a46a:::obj167:head 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2260a46a:::obj167:head v 107'167 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2f9dd5a:::obj121:head) v3 finish latency 0.000022 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2222704f:::obj161:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2260a46a:::obj167:head, version: 107'167, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2260a46a:::obj167:head@107'167, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2222704f:::obj161:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2222704f:::obj161:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 113/113 lb 4:2222704f:::obj161:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2222704f:::obj161:head) v3 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:212e6483:::obj209:head local-lis/les=112/113 n=98 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2222704f:::obj161:head) 2026-02-11T23:31:17.467+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2222704f:::obj161:head) v3 finish latency 0.000011 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.467+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d7800 con 0x3752000 2026-02-11T23:31:17.467+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f2fc2104:::obj429:head)]) -- 0x497e480 con 0x3752000 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.467+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.467+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2260a46a:::obj167:head)]) -- 0x49b8c00 con 0x3e4a000 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1187 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3752000 2026-02-11T23:31:17.467+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.467+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.467+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1188 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f2fc2104:::obj429:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5080 con 0x3752000 2026-02-11T23:31:17.467+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2fc2104:::obj429:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2fc2104:::obj429:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 113/109 [PushOp(1:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2fc2104:::obj429:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f31cde5e:::obj446:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f31cde5e:::obj446:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f31cde5e:::obj446:head 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f31cde5e:::obj446:head v 107'446 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.467+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f2fc2104:::obj429:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2fc2104:::obj429:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 1.0 e 113/113 lb 1:f2fc2104:::obj429:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f2fc2104:::obj429:head) v3 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2f9dd5a:::obj121:head local-lis/les=109/110 n=573 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f2fc2104:::obj429:head) 2026-02-11T23:31:17.467+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f2fc2104:::obj429:head) v3 finish latency 0.000012 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 245 ==== MOSDPGPush(4.0 113/112 [PushOp(4:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.467+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.467+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 246 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2260a46a:::obj167:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3e4a000 2026-02-11T23:31:17.467+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2260a46a:::obj167:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.467+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2260a46a:::obj167:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 113/112 [PushOp(4:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2260a46a:::obj167:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:233a42c1:::obj6:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:233a42c1:::obj6:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:233a42c1:::obj6:head 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:233a42c1:::obj6:head v 107'6 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.467+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.467+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:233a42c1:::obj6:head, version: 107'6, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:233a42c1:::obj6:head@107'6, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000041 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2260a46a:::obj167:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2260a46a:::obj167:head) v3 prio 15 cost 0 latency 0.000058 pg_backfill(progress 4.0 e 113/113 lb 4:2260a46a:::obj167:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2260a46a:::obj167:head) v3 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2222704f:::obj161:head local-lis/les=112/113 n=99 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2260a46a:::obj167:head) 2026-02-11T23:31:17.467+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2260a46a:::obj167:head) v3 finish latency 0.000010 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.467+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d7200 con 0x3752000 2026-02-11T23:31:17.467+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f31cde5e:::obj446:head)]) -- 0x4fce900 con 0x3752000 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.467+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.467+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96f00 con 0x3e4a000 2026-02-11T23:31:17.467+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:233a42c1:::obj6:head)]) -- 0x497e180 con 0x3e4a000 2026-02-11T23:31:17.467+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.468+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1189 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3752000 2026-02-11T23:31:17.468+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.468+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1190 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f31cde5e:::obj446:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3752000 2026-02-11T23:31:17.468+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f31cde5e:::obj446:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 113 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.468+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f31cde5e:::obj446:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 113/109 [PushOp(1:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f31cde5e:::obj446:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f33597d0:::obj489:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f33597d0:::obj489:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f33597d0:::obj489:head 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f33597d0:::obj489:head v 107'489 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.468+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f31cde5e:::obj446:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f31cde5e:::obj446:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 1.0 e 113/113 lb 1:f31cde5e:::obj446:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f31cde5e:::obj446:head) v3 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f2fc2104:::obj429:head local-lis/les=109/110 n=574 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f31cde5e:::obj446:head) 2026-02-11T23:31:17.468+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f31cde5e:::obj446:head) v3 finish latency 0.000014 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.468+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 247 ==== MOSDPGPush(4.0 113/112 [PushOp(4:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea1a0 con 0x3e4a000 2026-02-11T23:31:17.468+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.468+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.468+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d7080 con 0x3752000 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f33597d0:::obj489:head)]) -- 0x49d7380 con 0x3752000 2026-02-11T23:31:17.468+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 248 ==== pg_backfill(progress 4.0 e 113/113 lb 4:233a42c1:::obj6:head) ==== 1060+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3e4a000 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.468+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:233a42c1:::obj6:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.468+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:233a42c1:::obj6:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000042 MOSDPGPush(4.0 113/112 [PushOp(4:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:233a42c1:::obj6:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:239f544e:::obj128:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:239f544e:::obj128:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:239f544e:::obj128:head 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:239f544e:::obj128:head v 107'128 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.468+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:239f544e:::obj128:head, version: 107'128, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:239f544e:::obj128:head@107'128, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:233a42c1:::obj6:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:233a42c1:::obj6:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 113/113 lb 4:233a42c1:::obj6:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:233a42c1:::obj6:head) v3 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2260a46a:::obj167:head local-lis/les=112/113 n=100 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:233a42c1:::obj6:head) 2026-02-11T23:31:17.468+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:233a42c1:::obj6:head) v3 finish latency 0.000014 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.468+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96c00 con 0x3e4a000 2026-02-11T23:31:17.468+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:239f544e:::obj128:head)]) -- 0x4fce780 con 0x3e4a000 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.468+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1191 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3752000 2026-02-11T23:31:17.468+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.468+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1192 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f33597d0:::obj489:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3752000 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.468+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f33597d0:::obj489:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.468+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f33597d0:::obj489:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(1.0 113/109 [PushOp(1:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f33597d0:::obj489:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f3768836:::obj189:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f3768836:::obj189:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f3768836:::obj189:head 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f3768836:::obj189:head v 107'189 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.468+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.468+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.468+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f33597d0:::obj489:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f33597d0:::obj489:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 1.0 e 113/113 lb 1:f33597d0:::obj489:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f33597d0:::obj489:head) v3 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f31cde5e:::obj446:head local-lis/les=109/110 n=575 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f33597d0:::obj489:head) 2026-02-11T23:31:17.468+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.468+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f33597d0:::obj489:head) v3 finish latency 0.000013 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.468+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.469+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d6c00 con 0x3752000 2026-02-11T23:31:17.469+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f3768836:::obj189:head)]) -- 0x4c96f00 con 0x3752000 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 249 ==== MOSDPGPush(4.0 113/112 [PushOp(4:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516dd40 con 0x3e4a000 2026-02-11T23:31:17.469+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.469+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 250 ==== pg_backfill(progress 4.0 e 113/113 lb 4:239f544e:::obj128:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3e4a000 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.469+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:239f544e:::obj128:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:239f544e:::obj128:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:239f544e:::obj128:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:242c40e0:::obj23:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:242c40e0:::obj23:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:242c40e0:::obj23:head 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:242c40e0:::obj23:head v 107'23 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.469+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:242c40e0:::obj23:head, version: 107'23, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:242c40e0:::obj23:head@107'23, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:239f544e:::obj128:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:239f544e:::obj128:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 113/113 lb 4:239f544e:::obj128:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:239f544e:::obj128:head) v3 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:233a42c1:::obj6:head local-lis/les=112/113 n=101 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:239f544e:::obj128:head) 2026-02-11T23:31:17.469+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:239f544e:::obj128:head) v3 finish latency 0.000011 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.469+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96a80 con 0x3e4a000 2026-02-11T23:31:17.469+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:242c40e0:::obj23:head)]) -- 0x49d7200 con 0x3e4a000 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1193 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183380 con 0x3752000 2026-02-11T23:31:17.469+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.469+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1194 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f3768836:::obj189:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3752000 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.469+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f3768836:::obj189:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f3768836:::obj189:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f3768836:::obj189:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f3c2593a:::obj26:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f3c2593a:::obj26:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f3c2593a:::obj26:head 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f3c2593a:::obj26:head v 107'26 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.469+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f3768836:::obj189:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f3768836:::obj189:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 1.0 e 113/113 lb 1:f3768836:::obj189:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f3768836:::obj189:head) v3 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f33597d0:::obj489:head local-lis/les=109/110 n=576 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f3768836:::obj189:head) 2026-02-11T23:31:17.469+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f3768836:::obj189:head) v3 finish latency 0.000011 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.469+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4916000 con 0x3752000 2026-02-11T23:31:17.469+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f3c2593a:::obj26:head)]) -- 0x49b8c00 con 0x3752000 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 251 ==== MOSDPGPush(4.0 113/112 [PushOp(4:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de44e0 con 0x3e4a000 2026-02-11T23:31:17.469+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.469+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 252 ==== pg_backfill(progress 4.0 e 113/113 lb 4:242c40e0:::obj23:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3e4a000 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.469+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:242c40e0:::obj23:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 113 2026-02-11T23:31:17.469+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:242c40e0:::obj23:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 113/112 [PushOp(4:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:242c40e0:::obj23:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:249a23dc:::obj422:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:249a23dc:::obj422:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:249a23dc:::obj422:head 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:249a23dc:::obj422:head v 107'422 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.469+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.469+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:249a23dc:::obj422:head, version: 107'422, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:249a23dc:::obj422:head@107'422, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:242c40e0:::obj23:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:242c40e0:::obj23:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 113/113 lb 4:242c40e0:::obj23:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:242c40e0:::obj23:head) v3 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:239f544e:::obj128:head local-lis/les=112/113 n=102 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:242c40e0:::obj23:head) 2026-02-11T23:31:17.469+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.469+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:242c40e0:::obj23:head) v3 finish latency 0.000011 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.469+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.469+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.470+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96900 con 0x3e4a000 2026-02-11T23:31:17.470+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:249a23dc:::obj422:head)]) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1195 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d040 con 0x3752000 2026-02-11T23:31:17.470+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.470+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1196 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f3c2593a:::obj26:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3752000 2026-02-11T23:31:17.470+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f3c2593a:::obj26:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.470+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f3c2593a:::obj26:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f3c2593a:::obj26:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 113/109 [PushOp(1:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f42cbfe2:::obj212:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f42cbfe2:::obj212:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f42cbfe2:::obj212:head 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f42cbfe2:::obj212:head v 107'212 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.470+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f3c2593a:::obj26:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f3c2593a:::obj26:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 113/113 lb 1:f3c2593a:::obj26:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f3c2593a:::obj26:head) v3 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3768836:::obj189:head local-lis/les=109/110 n=577 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f3c2593a:::obj26:head) 2026-02-11T23:31:17.470+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f3c2593a:::obj26:head) v3 finish latency 0.000011 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.470+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49d6f00 con 0x3752000 2026-02-11T23:31:17.470+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f42cbfe2:::obj212:head)]) -- 0x49d7800 con 0x3752000 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 253 ==== MOSDPGPush(4.0 113/112 [PushOp(4:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.470+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.470+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 254 ==== pg_backfill(progress 4.0 e 113/113 lb 4:249a23dc:::obj422:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3e4a000 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.470+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:249a23dc:::obj422:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:249a23dc:::obj422:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:249a23dc:::obj422:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:24b5611b:::obj20:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:24b5611b:::obj20:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:24b5611b:::obj20:head 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:24b5611b:::obj20:head v 107'20 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.470+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:24b5611b:::obj20:head, version: 107'20, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24b5611b:::obj20:head@107'20, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:249a23dc:::obj422:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:249a23dc:::obj422:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 113/113 lb 4:249a23dc:::obj422:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:249a23dc:::obj422:head) v3 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:242c40e0:::obj23:head local-lis/les=112/113 n=103 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:249a23dc:::obj422:head) 2026-02-11T23:31:17.470+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:249a23dc:::obj422:head) v3 finish latency 0.000011 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.470+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96780 con 0x3e4a000 2026-02-11T23:31:17.470+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:24b5611b:::obj20:head)]) -- 0x4c97200 con 0x3e4a000 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1197 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d6c0 con 0x3752000 2026-02-11T23:31:17.470+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.470+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1198 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f42cbfe2:::obj212:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3752000 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.470+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f42cbfe2:::obj212:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.470+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f42cbfe2:::obj212:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f42cbfe2:::obj212:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f4af07f1:::obj515:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f4af07f1:::obj515:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f4af07f1:::obj515:head 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f4af07f1:::obj515:head v 107'515 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.470+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.470+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f42cbfe2:::obj212:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f42cbfe2:::obj212:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 1.0 e 113/113 lb 1:f42cbfe2:::obj212:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f42cbfe2:::obj212:head) v3 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f3c2593a:::obj26:head local-lis/les=109/110 n=578 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f42cbfe2:::obj212:head) 2026-02-11T23:31:17.470+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f42cbfe2:::obj212:head) v3 finish latency 0.000011 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.470+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.470+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4a04900 con 0x3752000 2026-02-11T23:31:17.470+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f4af07f1:::obj515:head)]) -- 0x49b8f00 con 0x3752000 2026-02-11T23:31:17.470+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.471+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 255 ==== MOSDPGPush(4.0 113/112 [PushOp(4:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.471+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.471+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 256 ==== pg_backfill(progress 4.0 e 113/113 lb 4:24b5611b:::obj20:head) ==== 1061+0+0 (unknown 0 0 0) 0x4ded080 con 0x3e4a000 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.471+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:24b5611b:::obj20:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 113 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:24b5611b:::obj20:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:24b5611b:::obj20:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 113/112 [PushOp(4:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:24fc6e92:::obj17:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:24fc6e92:::obj17:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:24fc6e92:::obj17:head 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:24fc6e92:::obj17:head v 107'17 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.471+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:24fc6e92:::obj17:head, version: 107'17, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:24fc6e92:::obj17:head@107'17, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:24b5611b:::obj20:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:24b5611b:::obj20:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 113/113 lb 4:24b5611b:::obj20:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:24b5611b:::obj20:head) v3 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:249a23dc:::obj422:head local-lis/les=112/113 n=104 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:24b5611b:::obj20:head) 2026-02-11T23:31:17.471+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:24b5611b:::obj20:head) v3 finish latency 0.000012 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.471+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1199 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3752000 2026-02-11T23:31:17.471+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.471+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1200 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f4af07f1:::obj515:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3752000 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.471+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f4af07f1:::obj515:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.471+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f4af07f1:::obj515:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 113/109 [PushOp(1:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f4af07f1:::obj515:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f4b32ae1:::obj130:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f4b32ae1:::obj130:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f4b32ae1:::obj130:head 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f4b32ae1:::obj130:head v 107'130 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.471+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000038 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f4af07f1:::obj515:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f4af07f1:::obj515:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 1.0 e 113/113 lb 1:f4af07f1:::obj515:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f4af07f1:::obj515:head) v3 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f42cbfe2:::obj212:head local-lis/les=109/110 n=579 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f4af07f1:::obj515:head) 2026-02-11T23:31:17.471+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f4af07f1:::obj515:head) v3 finish latency 0.000011 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.471+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96600 con 0x3e4a000 2026-02-11T23:31:17.471+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:24fc6e92:::obj17:head)]) -- 0x4fcf200 con 0x3e4a000 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.471+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4910180 con 0x3752000 2026-02-11T23:31:17.471+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f4b32ae1:::obj130:head)]) -- 0x4c97500 con 0x3752000 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.471+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 257 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ebba0 con 0x3e4a000 2026-02-11T23:31:17.471+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.471+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 258 ==== pg_backfill(progress 4.0 e 113/113 lb 4:24fc6e92:::obj17:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dee100 con 0x3e4a000 2026-02-11T23:31:17.471+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:24fc6e92:::obj17:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.471+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:24fc6e92:::obj17:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 113/112 [PushOp(4:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:24fc6e92:::obj17:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2540f982:::obj403:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2540f982:::obj403:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2540f982:::obj403:head 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2540f982:::obj403:head v 107'403 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.471+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2540f982:::obj403:head, version: 107'403, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2540f982:::obj403:head@107'403, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:24fc6e92:::obj17:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:24fc6e92:::obj17:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 113/113 lb 4:24fc6e92:::obj17:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:24fc6e92:::obj17:head) v3 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24b5611b:::obj20:head local-lis/les=112/113 n=105 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:24fc6e92:::obj17:head) 2026-02-11T23:31:17.471+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:24fc6e92:::obj17:head) v3 finish latency 0.000011 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.471+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1201 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3752000 2026-02-11T23:31:17.471+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.471+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1202 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f4b32ae1:::obj130:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3752000 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.471+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f4b32ae1:::obj130:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.471+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f4b32ae1:::obj130:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 113/109 [PushOp(1:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.471+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.471+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96480 con 0x3e4a000 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.471+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2540f982:::obj403:head)]) -- 0x4a04c00 con 0x3e4a000 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f677cfaa:::obj144:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.471+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f4b32ae1:::obj130:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.471+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f677cfaa:::obj144:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f677cfaa:::obj144:head 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f677cfaa:::obj144:head v 107'144 2026-02-11T23:31:17.471+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.471+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f4b32ae1:::obj130:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f4b32ae1:::obj130:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 113/113 lb 1:f4b32ae1:::obj130:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f4b32ae1:::obj130:head) v3 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4af07f1:::obj515:head local-lis/les=109/110 n=580 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f4b32ae1:::obj130:head) 2026-02-11T23:31:17.472+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f4b32ae1:::obj130:head) v3 finish latency 0.000011 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.472+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x49b8780 con 0x3752000 2026-02-11T23:31:17.472+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f677cfaa:::obj144:head)]) -- 0x4c97680 con 0x3752000 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.472+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 259 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3e4a000 2026-02-11T23:31:17.472+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.472+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.472+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 260 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2540f982:::obj403:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3e4a000 2026-02-11T23:31:17.472+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2540f982:::obj403:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.472+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2540f982:::obj403:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2540f982:::obj403:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2551b5e8:::obj231:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2551b5e8:::obj231:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2551b5e8:::obj231:head 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2551b5e8:::obj231:head v 107'231 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.472+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2551b5e8:::obj231:head, version: 107'231, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2551b5e8:::obj231:head@107'231, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2540f982:::obj403:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2540f982:::obj403:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 113/113 lb 4:2540f982:::obj403:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2540f982:::obj403:head) v3 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:24fc6e92:::obj17:head local-lis/les=112/113 n=106 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2540f982:::obj403:head) 2026-02-11T23:31:17.472+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2540f982:::obj403:head) v3 finish latency 0.000011 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.472+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1203 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3752000 2026-02-11T23:31:17.472+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.472+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.472+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1204 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f677cfaa:::obj144:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd180 con 0x3752000 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.472+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f677cfaa:::obj144:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.472+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f677cfaa:::obj144:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(1.0 113/109 [PushOp(1:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f677cfaa:::obj144:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f76d37c8:::obj371:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f76d37c8:::obj371:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f76d37c8:::obj371:head 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f76d37c8:::obj371:head v 107'371 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.472+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.472+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000039 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f677cfaa:::obj144:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f677cfaa:::obj144:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 1.0 e 113/113 lb 1:f677cfaa:::obj144:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f677cfaa:::obj144:head) v3 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f4b32ae1:::obj130:head local-lis/les=109/110 n=581 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f677cfaa:::obj144:head) 2026-02-11T23:31:17.472+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f677cfaa:::obj144:head) v3 finish latency 0.000012 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.472+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96300 con 0x3e4a000 2026-02-11T23:31:17.472+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2551b5e8:::obj231:head)]) -- 0x4eabc80 con 0x3e4a000 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.472+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.472+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.472+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4910000 con 0x3752000 2026-02-11T23:31:17.472+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f76d37c8:::obj371:head)]) -- 0x4c97800 con 0x3752000 2026-02-11T23:31:17.472+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.473+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1205 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3752000 2026-02-11T23:31:17.473+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.473+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1206 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f76d37c8:::obj371:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def180 con 0x3752000 2026-02-11T23:31:17.473+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f76d37c8:::obj371:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 113 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.473+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f76d37c8:::obj371:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 113/109 [PushOp(1:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f76d37c8:::obj371:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.473+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 261 ==== MOSDPGPush(4.0 113/112 [PushOp(4:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2340 con 0x3e4a000 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.473+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.473+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f77ac86f:::obj198:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f77ac86f:::obj198:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.473+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 262 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2551b5e8:::obj231:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3e4a000 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f77ac86f:::obj198:head 2026-02-11T23:31:17.473+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2551b5e8:::obj231:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f77ac86f:::obj198:head v 107'198 2026-02-11T23:31:17.473+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2551b5e8:::obj231:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.473+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000060 MOSDPGPush(4.0 113/112 [PushOp(4:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f76d37c8:::obj371:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f76d37c8:::obj371:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 1.0 e 113/113 lb 1:f76d37c8:::obj371:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f76d37c8:::obj371:head) v3 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f677cfaa:::obj144:head local-lis/les=109/110 n=582 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f76d37c8:::obj371:head) 2026-02-11T23:31:17.473+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f76d37c8:::obj371:head) v3 finish latency 0.000011 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2551b5e8:::obj231:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:26b5008c:::obj336:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:26b5008c:::obj336:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:26b5008c:::obj336:head 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:26b5008c:::obj336:head v 107'336 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.473+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:26b5008c:::obj336:head, version: 107'336, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26b5008c:::obj336:head@107'336, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2551b5e8:::obj231:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2551b5e8:::obj231:head) v3 prio 15 cost 0 latency 0.000100 pg_backfill(progress 4.0 e 113/113 lb 4:2551b5e8:::obj231:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2551b5e8:::obj231:head) v3 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2540f982:::obj403:head local-lis/les=112/113 n=107 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2551b5e8:::obj231:head) 2026-02-11T23:31:17.473+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2551b5e8:::obj231:head) v3 finish latency 0.000010 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.473+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7c000 con 0x3752000 2026-02-11T23:31:17.473+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f77ac86f:::obj198:head)]) -- 0x4c96180 con 0x3752000 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.473+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7c180 con 0x3e4a000 2026-02-11T23:31:17.473+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:26b5008c:::obj336:head)]) -- 0x4c97980 con 0x3e4a000 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.473+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1207 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43eba00 con 0x3752000 2026-02-11T23:31:17.473+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.473+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1208 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f77ac86f:::obj198:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164c00 con 0x3752000 2026-02-11T23:31:17.473+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f77ac86f:::obj198:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 113 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.473+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f77ac86f:::obj198:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(1.0 113/109 [PushOp(1:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f77ac86f:::obj198:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f7c5b032:::obj82:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f7c5b032:::obj82:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f7c5b032:::obj82:head 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f7c5b032:::obj82:head v 107'82 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.473+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.473+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 263 ==== MOSDPGPush(4.0 113/112 [PushOp(4:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.473+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.473+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.473+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 264 ==== pg_backfill(progress 4.0 e 113/113 lb 4:26b5008c:::obj336:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165180 con 0x3e4a000 2026-02-11T23:31:17.473+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:26b5008c:::obj336:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.473+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:26b5008c:::obj336:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f77ac86f:::obj198:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 113/112 [PushOp(4:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f77ac86f:::obj198:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 1.0 e 113/113 lb 1:f77ac86f:::obj198:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f77ac86f:::obj198:head) v3 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f76d37c8:::obj371:head local-lis/les=109/110 n=583 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f77ac86f:::obj198:head) 2026-02-11T23:31:17.473+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f77ac86f:::obj198:head) v3 finish latency 0.000011 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:26b5008c:::obj336:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:26cf9b2d:::obj546:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:26cf9b2d:::obj546:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:26cf9b2d:::obj546:head 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:26cf9b2d:::obj546:head v 107'546 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.473+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.473+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:26cf9b2d:::obj546:head, version: 107'546, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:26cf9b2d:::obj546:head@107'546, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:26b5008c:::obj336:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:26b5008c:::obj336:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 113/113 lb 4:26b5008c:::obj336:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:26b5008c:::obj336:head) v3 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2551b5e8:::obj231:head local-lis/les=112/113 n=108 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:26b5008c:::obj336:head) 2026-02-11T23:31:17.473+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:26b5008c:::obj336:head) v3 finish latency 0.000010 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.473+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.473+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.473+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7c300 con 0x3752000 2026-02-11T23:31:17.473+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f7c5b032:::obj82:head)]) -- 0x4c97e00 con 0x3752000 2026-02-11T23:31:17.473+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.474+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7c480 con 0x3e4a000 2026-02-11T23:31:17.474+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:26cf9b2d:::obj546:head)]) -- 0x4dd6780 con 0x3e4a000 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1209 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3752000 2026-02-11T23:31:17.474+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.474+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1210 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f7c5b032:::obj82:head) ==== 1061+0+0 (unknown 0 0 0) 0x5165700 con 0x3752000 2026-02-11T23:31:17.474+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f7c5b032:::obj82:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 113 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.474+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f7c5b032:::obj82:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 113/109 [PushOp(1:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f7c5b032:::obj82:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f7eaa9b8:::obj104:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f7eaa9b8:::obj104:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f7eaa9b8:::obj104:head 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f7eaa9b8:::obj104:head v 107'104 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.474+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f7c5b032:::obj82:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f7c5b032:::obj82:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 1.0 e 113/113 lb 1:f7c5b032:::obj82:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f7c5b032:::obj82:head) v3 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f77ac86f:::obj198:head local-lis/les=109/110 n=584 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f7c5b032:::obj82:head) 2026-02-11T23:31:17.474+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f7c5b032:::obj82:head) v3 finish latency 0.000011 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.474+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7c600 con 0x3752000 2026-02-11T23:31:17.474+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f7eaa9b8:::obj104:head)]) -- 0x4fcfe00 con 0x3752000 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 265 ==== MOSDPGPush(4.0 113/112 [PushOp(4:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.474+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.474+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.474+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 266 ==== pg_backfill(progress 4.0 e 113/113 lb 4:26cf9b2d:::obj546:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:26cf9b2d:::obj546:head) v3 prio 15 type 95 cost 0 latency 0.000014 epoch 113 2026-02-11T23:31:17.474+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:26cf9b2d:::obj546:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 113/112 [PushOp(4:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:26cf9b2d:::obj546:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:27794e2d:::obj115:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:27794e2d:::obj115:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:27794e2d:::obj115:head 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:27794e2d:::obj115:head v 107'115 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.474+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:27794e2d:::obj115:head, version: 107'115, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27794e2d:::obj115:head@107'115, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:26cf9b2d:::obj546:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:26cf9b2d:::obj546:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 113/113 lb 4:26cf9b2d:::obj546:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:26cf9b2d:::obj546:head) v3 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26b5008c:::obj336:head local-lis/les=112/113 n=109 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:26cf9b2d:::obj546:head) 2026-02-11T23:31:17.474+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:26cf9b2d:::obj546:head) v3 finish latency 0.000011 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.474+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7c780 con 0x3e4a000 2026-02-11T23:31:17.474+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:27794e2d:::obj115:head)]) -- 0x4c96000 con 0x3e4a000 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1211 ==== MOSDPGPush(1.0 113/109 [PushOp(1:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3752000 2026-02-11T23:31:17.474+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.474+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1212 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f7eaa9b8:::obj104:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3752000 2026-02-11T23:31:17.474+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f7eaa9b8:::obj104:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.474+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f7eaa9b8:::obj104:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f7eaa9b8:::obj104:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(1.0 113/109 [PushOp(1:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:f948044a:::obj145:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:f948044a:::obj145:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:f948044a:::obj145:head 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:f948044a:::obj145:head v 107'145 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.474+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.474+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.474+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f7eaa9b8:::obj104:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f7eaa9b8:::obj104:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 113/113 lb 1:f7eaa9b8:::obj104:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f7eaa9b8:::obj104:head) v3 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7c5b032:::obj82:head local-lis/les=109/110 n=585 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f7eaa9b8:::obj104:head) 2026-02-11T23:31:17.474+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.474+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f7eaa9b8:::obj104:head) v3 finish latency 0.000012 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.474+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.475+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7c900 con 0x3752000 2026-02-11T23:31:17.475+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:f948044a:::obj145:head)]) -- 0x4c97800 con 0x3752000 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 267 ==== MOSDPGPush(4.0 113/112 [PushOp(4:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183040 con 0x3e4a000 2026-02-11T23:31:17.475+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.475+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.475+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 268 ==== pg_backfill(progress 4.0 e 113/113 lb 4:27794e2d:::obj115:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.475+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:27794e2d:::obj115:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:27794e2d:::obj115:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 113/112 [PushOp(4:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:27794e2d:::obj115:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:27952899:::obj201:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:27952899:::obj201:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:27952899:::obj201:head 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:27952899:::obj201:head v 107'201 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.475+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:27952899:::obj201:head, version: 107'201, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:27952899:::obj201:head@107'201, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:27794e2d:::obj115:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:27794e2d:::obj115:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 113/113 lb 4:27794e2d:::obj115:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:27794e2d:::obj115:head) v3 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:26cf9b2d:::obj546:head local-lis/les=112/113 n=110 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:27794e2d:::obj115:head) 2026-02-11T23:31:17.475+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:27794e2d:::obj115:head) v3 finish latency 0.000012 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.475+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7ca80 con 0x3e4a000 2026-02-11T23:31:17.475+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:27952899:::obj201:head)]) -- 0x4eabc80 con 0x3e4a000 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1213 ==== MOSDPGPush(1.0 113/109 [PushOp(1:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4680 con 0x3752000 2026-02-11T23:31:17.475+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.475+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(1.0 113/109 [PushOp(1:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.475+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1214 ==== pg_backfill(progress 1.0 e 113/113 lb 1:f948044a:::obj145:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3752000 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.475+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:f948044a:::obj145:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.475+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f948044a:::obj145:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:fa02d04b:::obj266:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f948044a:::obj145:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:fa02d04b:::obj266:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:fa02d04b:::obj266:head 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:fa02d04b:::obj266:head v 107'266 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.475+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:f948044a:::obj145:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f948044a:::obj145:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 113/113 lb 1:f948044a:::obj145:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:f948044a:::obj145:head) v3 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f7eaa9b8:::obj104:head local-lis/les=109/110 n=586 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:f948044a:::obj145:head) 2026-02-11T23:31:17.475+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:f948044a:::obj145:head) v3 finish latency 0.000012 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.475+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7cc00 con 0x3752000 2026-02-11T23:31:17.475+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:fa02d04b:::obj266:head)]) -- 0x4c97680 con 0x3752000 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 269 ==== MOSDPGPush(4.0 113/112 [PushOp(4:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516da00 con 0x3e4a000 2026-02-11T23:31:17.475+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.475+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 270 ==== pg_backfill(progress 4.0 e 113/113 lb 4:27952899:::obj201:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.475+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:27952899:::obj201:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.475+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:27952899:::obj201:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 113/112 [PushOp(4:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:27952899:::obj201:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:29cd40a5:::obj549:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:29cd40a5:::obj549:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:29cd40a5:::obj549:head 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:29cd40a5:::obj549:head v 107'549 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.475+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.475+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:29cd40a5:::obj549:head, version: 107'549, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:29cd40a5:::obj549:head@107'549, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.475+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:27952899:::obj201:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:27952899:::obj201:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 113/113 lb 4:27952899:::obj201:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:27952899:::obj201:head) v3 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27794e2d:::obj115:head local-lis/les=112/113 n=111 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:27952899:::obj201:head) 2026-02-11T23:31:17.475+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.475+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:27952899:::obj201:head) v3 finish latency 0.000011 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.475+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.475+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1215 ==== MOSDPGPush(1.0 113/109 [PushOp(1:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182000 con 0x3752000 2026-02-11T23:31:17.476+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 113 2026-02-11T23:31:17.476+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.476+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1216 ==== pg_backfill(progress 1.0 e 113/113 lb 1:fa02d04b:::obj266:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3752000 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:fa02d04b:::obj266:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.476+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fa02d04b:::obj266:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 113/109 [PushOp(1:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fa02d04b:::obj266:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:fa6c2b5c:::obj438:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:fa6c2b5c:::obj438:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:fa6c2b5c:::obj438:head 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:fa6c2b5c:::obj438:head v 107'438 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.476+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fa02d04b:::obj266:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fa02d04b:::obj266:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 1.0 e 113/113 lb 1:fa02d04b:::obj266:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.476+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7d080 con 0x3e4a000 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:fa02d04b:::obj266:head) v3 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:f948044a:::obj145:head local-lis/les=109/110 n=587 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:fa02d04b:::obj266:head) 2026-02-11T23:31:17.476+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.476+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:29cd40a5:::obj549:head)]) -- 0x4a04c00 con 0x3e4a000 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fa02d04b:::obj266:head) v3 finish latency 0.000011 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.476+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7d200 con 0x3752000 2026-02-11T23:31:17.476+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:fa6c2b5c:::obj438:head)]) -- 0x4b7cd80 con 0x3752000 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.476+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 271 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51824e0 con 0x3e4a000 2026-02-11T23:31:17.476+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.476+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 272 ==== pg_backfill(progress 4.0 e 113/113 lb 4:29cd40a5:::obj549:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.476+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:29cd40a5:::obj549:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 113 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:29cd40a5:::obj549:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:29cd40a5:::obj549:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2a627378:::obj100:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2a627378:::obj100:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2a627378:::obj100:head 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2a627378:::obj100:head v 107'100 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.476+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1217 ==== MOSDPGPush(1.0 113/109 [PushOp(1:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3752000 2026-02-11T23:31:17.476+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.476+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.476+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.476+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1218 ==== pg_backfill(progress 1.0 e 113/113 lb 1:fa6c2b5c:::obj438:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163b80 con 0x3752000 2026-02-11T23:31:17.476+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:fa6c2b5c:::obj438:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.476+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fa6c2b5c:::obj438:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2a627378:::obj100:head, version: 107'100, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2a627378:::obj100:head@107'100, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:29cd40a5:::obj549:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000043 MOSDPGPush(1.0 113/109 [PushOp(1:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:29cd40a5:::obj549:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 113/113 lb 4:29cd40a5:::obj549:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:29cd40a5:::obj549:head) v3 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:27952899:::obj201:head local-lis/les=112/113 n=112 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:29cd40a5:::obj549:head) 2026-02-11T23:31:17.476+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:29cd40a5:::obj549:head) v3 finish latency 0.000013 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:fb333d49:::obj180:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fa6c2b5c:::obj438:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:fb333d49:::obj180:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:fb333d49:::obj180:head 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:fb333d49:::obj180:head v 107'180 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.476+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.476+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fa6c2b5c:::obj438:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fa6c2b5c:::obj438:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 1.0 e 113/113 lb 1:fa6c2b5c:::obj438:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:fa6c2b5c:::obj438:head) v3 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa02d04b:::obj266:head local-lis/les=109/110 n=588 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:fa6c2b5c:::obj438:head) 2026-02-11T23:31:17.476+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fa6c2b5c:::obj438:head) v3 finish latency 0.000012 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.476+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7d380 con 0x3e4a000 2026-02-11T23:31:17.476+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2a627378:::obj100:head)]) -- 0x4c97500 con 0x3e4a000 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.476+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.476+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.476+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7d500 con 0x3752000 2026-02-11T23:31:17.476+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:fb333d49:::obj180:head)]) -- 0x4fcf200 con 0x3752000 2026-02-11T23:31:17.476+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.477+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1219 ==== MOSDPGPush(1.0 113/109 [PushOp(1:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2680 con 0x3752000 2026-02-11T23:31:17.477+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.477+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1220 ==== pg_backfill(progress 1.0 e 113/113 lb 1:fb333d49:::obj180:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163600 con 0x3752000 2026-02-11T23:31:17.477+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:fb333d49:::obj180:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.477+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fb333d49:::obj180:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(1.0 113/109 [PushOp(1:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fb333d49:::obj180:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:fb44f156:::obj279:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:fb44f156:::obj279:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.477+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 273 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:fb44f156:::obj279:head 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:fb44f156:::obj279:head v 107'279 2026-02-11T23:31:17.477+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.477+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.477+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.477+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 274 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2a627378:::obj100:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163080 con 0x3e4a000 2026-02-11T23:31:17.477+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2a627378:::obj100:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.477+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2a627378:::obj100:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fb333d49:::obj180:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000052 MOSDPGPush(4.0 113/112 [PushOp(4:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fb333d49:::obj180:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 113/113 lb 1:fb333d49:::obj180:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:fb333d49:::obj180:head) v3 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fa6c2b5c:::obj438:head local-lis/les=109/110 n=589 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:fb333d49:::obj180:head) 2026-02-11T23:31:17.477+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2acce685:::obj583:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2acce685:::obj583:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fb333d49:::obj180:head) v3 finish latency 0.000011 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2a627378:::obj100:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2acce685:::obj583:head 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2acce685:::obj583:head v 107'583 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.477+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2acce685:::obj583:head, version: 107'583, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2acce685:::obj583:head@107'583, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2a627378:::obj100:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2a627378:::obj100:head) v3 prio 15 cost 0 latency 0.000096 pg_backfill(progress 4.0 e 113/113 lb 4:2a627378:::obj100:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2a627378:::obj100:head) v3 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:29cd40a5:::obj549:head local-lis/les=112/113 n=113 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2a627378:::obj100:head) 2026-02-11T23:31:17.477+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2a627378:::obj100:head) v3 finish latency 0.000011 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.477+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7d680 con 0x3752000 2026-02-11T23:31:17.477+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:fb44f156:::obj279:head)]) -- 0x49b8f00 con 0x3752000 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.477+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7d800 con 0x3e4a000 2026-02-11T23:31:17.477+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2acce685:::obj583:head)]) -- 0x4c97200 con 0x3e4a000 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.477+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1221 ==== MOSDPGPush(1.0 113/109 [PushOp(1:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3752000 2026-02-11T23:31:17.477+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.477+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1222 ==== pg_backfill(progress 1.0 e 113/113 lb 1:fb44f156:::obj279:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3752000 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.477+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:fb44f156:::obj279:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.477+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fb44f156:::obj279:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fb44f156:::obj279:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:fb8f5c57:::obj248:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:fb8f5c57:::obj248:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:fb8f5c57:::obj248:head 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:fb8f5c57:::obj248:head v 107'248 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.477+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fb44f156:::obj279:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fb44f156:::obj279:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 1.0 e 113/113 lb 1:fb44f156:::obj279:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:fb44f156:::obj279:head) v3 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb333d49:::obj180:head local-lis/les=109/110 n=590 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:fb44f156:::obj279:head) 2026-02-11T23:31:17.477+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.477+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fb44f156:::obj279:head) v3 finish latency 0.000016 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.477+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.477+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7dc80 con 0x3752000 2026-02-11T23:31:17.477+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:fb8f5c57:::obj248:head)]) -- 0x4b7d980 con 0x3752000 2026-02-11T23:31:17.477+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.477+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 275 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.477+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.477+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.477+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 276 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2acce685:::obj583:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162580 con 0x3e4a000 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.477+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2acce685:::obj583:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 113 2026-02-11T23:31:17.477+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.477+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2acce685:::obj583:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2acce685:::obj583:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 113/112 [PushOp(4:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2b184b8d:::obj560:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2b184b8d:::obj560:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2b184b8d:::obj560:head 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2b184b8d:::obj560:head v 107'560 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.478+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2b184b8d:::obj560:head, version: 107'560, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2b184b8d:::obj560:head@107'560, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2acce685:::obj583:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2acce685:::obj583:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 113/113 lb 4:2acce685:::obj583:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2acce685:::obj583:head) v3 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2a627378:::obj100:head local-lis/les=112/113 n=114 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2acce685:::obj583:head) 2026-02-11T23:31:17.478+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2acce685:::obj583:head) v3 finish latency 0.000014 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.478+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7de00 con 0x3e4a000 2026-02-11T23:31:17.478+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2b184b8d:::obj560:head)]) -- 0x49d7800 con 0x3e4a000 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1223 ==== MOSDPGPush(1.0 113/109 [PushOp(1:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3752000 2026-02-11T23:31:17.478+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.478+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.478+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1224 ==== pg_backfill(progress 1.0 e 113/113 lb 1:fb8f5c57:::obj248:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162000 con 0x3752000 2026-02-11T23:31:17.478+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:fb8f5c57:::obj248:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fb8f5c57:::obj248:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 113/109 [PushOp(1:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fb8f5c57:::obj248:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:fcc34b3b:::obj191:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:fcc34b3b:::obj191:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:fcc34b3b:::obj191:head 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:fcc34b3b:::obj191:head v 107'191 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.478+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fb8f5c57:::obj248:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fb8f5c57:::obj248:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 1.0 e 113/113 lb 1:fb8f5c57:::obj248:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:fb8f5c57:::obj248:head) v3 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb44f156:::obj279:head local-lis/les=109/110 n=591 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:fb8f5c57:::obj248:head) 2026-02-11T23:31:17.478+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fb8f5c57:::obj248:head) v3 finish latency 0.000012 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.478+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7de00 con 0x3752000 2026-02-11T23:31:17.478+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:fcc34b3b:::obj191:head)]) -- 0x4c97080 con 0x3752000 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 277 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2340 con 0x3e4a000 2026-02-11T23:31:17.478+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.478+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.478+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 278 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2b184b8d:::obj560:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3e4a000 2026-02-11T23:31:17.478+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2b184b8d:::obj560:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2b184b8d:::obj560:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 113/112 [PushOp(4:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2b184b8d:::obj560:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2c8d8a76:::obj467:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2c8d8a76:::obj467:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2c8d8a76:::obj467:head 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2c8d8a76:::obj467:head v 107'467 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.478+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2c8d8a76:::obj467:head, version: 107'467, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2c8d8a76:::obj467:head@107'467, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000041 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2b184b8d:::obj560:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2b184b8d:::obj560:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 4.0 e 113/113 lb 4:2b184b8d:::obj560:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2b184b8d:::obj560:head) v3 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2acce685:::obj583:head local-lis/les=112/113 n=115 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2b184b8d:::obj560:head) 2026-02-11T23:31:17.478+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2b184b8d:::obj560:head) v3 finish latency 0.000011 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.478+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7d980 con 0x3e4a000 2026-02-11T23:31:17.478+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2c8d8a76:::obj467:head)]) -- 0x49b8c00 con 0x3e4a000 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1225 ==== MOSDPGPush(1.0 113/109 [PushOp(1:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3752000 2026-02-11T23:31:17.478+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.478+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1226 ==== pg_backfill(progress 1.0 e 113/113 lb 1:fcc34b3b:::obj191:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3752000 2026-02-11T23:31:17.478+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:fcc34b3b:::obj191:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.478+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fcc34b3b:::obj191:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 113/109 [PushOp(1:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fcc34b3b:::obj191:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:fcdeb4b1:::obj379:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:fcdeb4b1:::obj379:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:fcdeb4b1:::obj379:head 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:fcdeb4b1:::obj379:head v 107'379 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.478+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.478+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.478+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.478+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fcc34b3b:::obj191:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fcc34b3b:::obj191:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 1.0 e 113/113 lb 1:fcc34b3b:::obj191:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:fcc34b3b:::obj191:head) v3 2026-02-11T23:31:17.478+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fb8f5c57:::obj248:head local-lis/les=109/110 n=592 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:fcc34b3b:::obj191:head) 2026-02-11T23:31:17.479+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fcc34b3b:::obj191:head) v3 finish latency 0.000016 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.479+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7dc80 con 0x3752000 2026-02-11T23:31:17.479+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:fcdeb4b1:::obj379:head)]) -- 0x49d7200 con 0x3752000 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 279 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3e4a000 2026-02-11T23:31:17.479+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.479+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.479+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 280 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2c8d8a76:::obj467:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3e4a000 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.479+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2c8d8a76:::obj467:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 113 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.479+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2c8d8a76:::obj467:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2c8d8a76:::obj467:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2caf4871:::obj405:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2caf4871:::obj405:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2caf4871:::obj405:head 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2caf4871:::obj405:head v 107'405 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.479+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2caf4871:::obj405:head, version: 107'405, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2caf4871:::obj405:head@107'405, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2c8d8a76:::obj467:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2c8d8a76:::obj467:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 113/113 lb 4:2c8d8a76:::obj467:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2c8d8a76:::obj467:head) v3 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2b184b8d:::obj560:head local-lis/les=112/113 n=116 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2c8d8a76:::obj467:head) 2026-02-11T23:31:17.479+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2c8d8a76:::obj467:head) v3 finish latency 0.000013 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.479+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c97200 con 0x3e4a000 2026-02-11T23:31:17.479+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2caf4871:::obj405:head)]) -- 0x4c96f00 con 0x3e4a000 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1227 ==== MOSDPGPush(1.0 113/109 [PushOp(1:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3752000 2026-02-11T23:31:17.479+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 113 2026-02-11T23:31:17.479+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.479+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1228 ==== pg_backfill(progress 1.0 e 113/113 lb 1:fcdeb4b1:::obj379:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3752000 2026-02-11T23:31:17.479+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:fcdeb4b1:::obj379:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.479+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fcdeb4b1:::obj379:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(1.0 113/109 [PushOp(1:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fcdeb4b1:::obj379:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:fd91bcb6:::obj528:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:fd91bcb6:::obj528:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:fd91bcb6:::obj528:head 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:fd91bcb6:::obj528:head v 107'528 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.479+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fcdeb4b1:::obj379:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fcdeb4b1:::obj379:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 1.0 e 113/113 lb 1:fcdeb4b1:::obj379:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:fcdeb4b1:::obj379:head) v3 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcc34b3b:::obj191:head local-lis/les=109/110 n=593 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:fcdeb4b1:::obj379:head) 2026-02-11T23:31:17.479+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fcdeb4b1:::obj379:head) v3 finish latency 0.000011 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.479+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7d800 con 0x3752000 2026-02-11T23:31:17.479+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:fd91bcb6:::obj528:head)]) -- 0x4fce780 con 0x3752000 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 281 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3e4a000 2026-02-11T23:31:17.479+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.479+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.479+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 282 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2caf4871:::obj405:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d700 con 0x3e4a000 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.479+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2caf4871:::obj405:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 113 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.479+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2caf4871:::obj405:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 113/112 [PushOp(4:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.479+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2caf4871:::obj405:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.479+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2d59679a:::obj179:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.479+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2d59679a:::obj179:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2d59679a:::obj179:head 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2d59679a:::obj179:head v 107'179 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.480+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2d59679a:::obj179:head, version: 107'179, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2d59679a:::obj179:head@107'179, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2caf4871:::obj405:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2caf4871:::obj405:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 113/113 lb 4:2caf4871:::obj405:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2caf4871:::obj405:head) v3 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2c8d8a76:::obj467:head local-lis/les=112/113 n=117 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2caf4871:::obj405:head) 2026-02-11T23:31:17.480+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2caf4871:::obj405:head) v3 finish latency 0.000011 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1229 ==== MOSDPGPush(1.0 113/109 [PushOp(1:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x43ebba0 con 0x3752000 2026-02-11T23:31:17.480+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.480+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.480+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1230 ==== pg_backfill(progress 1.0 e 113/113 lb 1:fd91bcb6:::obj528:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3752000 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:fd91bcb6:::obj528:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.480+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fd91bcb6:::obj528:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 113/109 [PushOp(1:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fd91bcb6:::obj528:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:fe351e27:::obj9:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:fe351e27:::obj9:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:fe351e27:::obj9:head 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:fe351e27:::obj9:head v 107'9 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.480+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fd91bcb6:::obj528:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fd91bcb6:::obj528:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 1.0 e 113/113 lb 1:fd91bcb6:::obj528:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:fd91bcb6:::obj528:head) v3 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fcdeb4b1:::obj379:head local-lis/les=109/110 n=594 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:fd91bcb6:::obj528:head) 2026-02-11T23:31:17.480+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fd91bcb6:::obj528:head) v3 finish latency 0.000010 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.480+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x49b8f00 con 0x3e4a000 2026-02-11T23:31:17.480+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2d59679a:::obj179:head)]) -- 0x49d7380 con 0x3e4a000 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.480+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7d680 con 0x3752000 2026-02-11T23:31:17.480+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:fe351e27:::obj9:head)]) -- 0x497e180 con 0x3752000 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1231 ==== MOSDPGPush(1.0 113/109 [PushOp(1:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183860 con 0x3752000 2026-02-11T23:31:17.480+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.480+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1232 ==== pg_backfill(progress 1.0 e 113/113 lb 1:fe351e27:::obj9:head) ==== 1060+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3752000 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.480+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:fe351e27:::obj9:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.480+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fe351e27:::obj9:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(1.0 113/109 [PushOp(1:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fe351e27:::obj9:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:fedf1386:::obj565:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:fedf1386:::obj565:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:fedf1386:::obj565:head 2026-02-11T23:31:17.480+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 283 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:fedf1386:::obj565:head v 107'565 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.480+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.480+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.480+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.480+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 284 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2d59679a:::obj179:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3e4a000 2026-02-11T23:31:17.480+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2d59679a:::obj179:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.480+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2d59679a:::obj179:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000049 MOSDPGPush(4.0 113/112 [PushOp(4:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fe351e27:::obj9:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fe351e27:::obj9:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 1.0 e 113/113 lb 1:fe351e27:::obj9:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:fe351e27:::obj9:head) v3 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fd91bcb6:::obj528:head local-lis/les=109/110 n=595 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:fe351e27:::obj9:head) 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2df8f997:::obj267:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2df8f997:::obj267:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2df8f997:::obj267:head 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2df8f997:::obj267:head v 107'267 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.480+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.480+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fe351e27:::obj9:head) v3 finish latency 0.000039 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2d59679a:::obj179:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.480+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2df8f997:::obj267:head, version: 107'267, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2df8f997:::obj267:head@107'267, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2d59679a:::obj179:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2d59679a:::obj179:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 4.0 e 113/113 lb 4:2d59679a:::obj179:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2d59679a:::obj179:head) v3 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2caf4871:::obj405:head local-lis/les=112/113 n=118 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2d59679a:::obj179:head) 2026-02-11T23:31:17.480+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2d59679a:::obj179:head) v3 finish latency 0.000011 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.480+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.480+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4fcf200 con 0x3752000 2026-02-11T23:31:17.480+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:fedf1386:::obj565:head)]) -- 0x4c80000 con 0x3752000 2026-02-11T23:31:17.480+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.481+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7d500 con 0x3e4a000 2026-02-11T23:31:17.481+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2df8f997:::obj267:head)]) -- 0x4c80300 con 0x3e4a000 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.481+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1233 ==== MOSDPGPush(1.0 113/109 [PushOp(1:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d6c0 con 0x3752000 2026-02-11T23:31:17.481+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.481+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.481+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1234 ==== pg_backfill(progress 1.0 e 113/113 lb 1:fedf1386:::obj565:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3752000 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:fedf1386:::obj565:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 113 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.481+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fedf1386:::obj565:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(1.0 113/109 [PushOp(1:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fedf1386:::obj565:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ff38ecdd:::obj155:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ff38ecdd:::obj155:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ff38ecdd:::obj155:head 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ff38ecdd:::obj155:head v 107'155 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.481+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:fedf1386:::obj565:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fedf1386:::obj565:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 1.0 e 113/113 lb 1:fedf1386:::obj565:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:fedf1386:::obj565:head) v3 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fe351e27:::obj9:head local-lis/les=109/110 n=596 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:fedf1386:::obj565:head) 2026-02-11T23:31:17.481+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:fedf1386:::obj565:head) v3 finish latency 0.000021 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.481+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 285 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.481+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.481+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.481+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 286 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2df8f997:::obj267:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3e4a000 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2df8f997:::obj267:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 113 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.481+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2df8f997:::obj267:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2df8f997:::obj267:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2e5f3a17:::obj520:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2e5f3a17:::obj520:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2e5f3a17:::obj520:head 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2e5f3a17:::obj520:head v 107'520 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.481+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2e5f3a17:::obj520:head, version: 107'520, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e5f3a17:::obj520:head@107'520, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2df8f997:::obj267:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2df8f997:::obj267:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 113/113 lb 4:2df8f997:::obj267:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2df8f997:::obj267:head) v3 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2d59679a:::obj179:head local-lis/les=112/113 n=119 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2df8f997:::obj267:head) 2026-02-11T23:31:17.481+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2df8f997:::obj267:head) v3 finish latency 0.000013 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.481+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4c97500 con 0x3752000 2026-02-11T23:31:17.481+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:ff38ecdd:::obj155:head)]) -- 0x4c80600 con 0x3752000 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.481+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7d380 con 0x3e4a000 2026-02-11T23:31:17.481+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2e5f3a17:::obj520:head)]) -- 0x4c80900 con 0x3e4a000 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.481+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1235 ==== MOSDPGPush(1.0 113/109 [PushOp(1:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3752000 2026-02-11T23:31:17.481+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 113 2026-02-11T23:31:17.481+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1236 ==== pg_backfill(progress 1.0 e 113/113 lb 1:ff38ecdd:::obj155:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3752000 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.481+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:ff38ecdd:::obj155:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 113 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ff38ecdd:::obj155:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(1.0 113/109 [PushOp(1:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ff38ecdd:::obj155:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ff7b1f36:::obj1:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ff7b1f36:::obj1:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ff7b1f36:::obj1:head 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ff7b1f36:::obj1:head v 107'1 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.481+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.481+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.481+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ff38ecdd:::obj155:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ff38ecdd:::obj155:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 1.0 e 113/113 lb 1:ff38ecdd:::obj155:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:ff38ecdd:::obj155:head) v3 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:fedf1386:::obj565:head local-lis/les=109/110 n=597 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:ff38ecdd:::obj155:head) 2026-02-11T23:31:17.481+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.481+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ff38ecdd:::obj155:head) v3 finish latency 0.000016 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.481+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 287 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3e4a000 2026-02-11T23:31:17.482+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.482+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 288 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2e5f3a17:::obj520:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3e4a000 2026-02-11T23:31:17.482+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2e5f3a17:::obj520:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.482+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2e5f3a17:::obj520:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 113/112 [PushOp(4:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2e5f3a17:::obj520:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2e69a8fc:::obj598:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2e69a8fc:::obj598:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2e69a8fc:::obj598:head 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2e69a8fc:::obj598:head v 107'598 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.482+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2e69a8fc:::obj598:head, version: 107'598, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e69a8fc:::obj598:head@107'598, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2e5f3a17:::obj520:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2e5f3a17:::obj520:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 113/113 lb 4:2e5f3a17:::obj520:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2e5f3a17:::obj520:head) v3 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2df8f997:::obj267:head local-lis/les=112/113 n=120 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2e5f3a17:::obj520:head) 2026-02-11T23:31:17.482+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2e5f3a17:::obj520:head) v3 finish latency 0.000011 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.482+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4b7cd80 con 0x3752000 2026-02-11T23:31:17.482+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:ff7b1f36:::obj1:head)]) -- 0x4c80c00 con 0x3752000 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.482+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7d200 con 0x3e4a000 2026-02-11T23:31:17.482+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2e69a8fc:::obj598:head)]) -- 0x4c80d80 con 0x3e4a000 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1237 ==== MOSDPGPush(1.0 113/109 [PushOp(1:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51821a0 con 0x3752000 2026-02-11T23:31:17.482+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(1.0 113/109 [PushOp(1:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 113 2026-02-11T23:31:17.482+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1238 ==== pg_backfill(progress 1.0 e 113/113 lb 1:ff7b1f36:::obj1:head) ==== 1060+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3752000 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.482+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 1.0 e 113/113 lb 1:ff7b1f36:::obj1:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ff7b1f36:::obj1:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=MOSDPGPush(1.0 113/109 [PushOp(1:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4a2a000 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(1.0 113/109 [PushOp(1:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ff7b1f36:::obj1:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(1.0 113/109 [PushOp(1:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 1:ffd8ed3d:::obj316:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 1:ffd8ed3d:::obj316:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 1:ffd8ed3d:::obj316:head 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 1:ffd8ed3d:::obj316:head v 107'316 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.482+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(1.0 113/109 [PushOp(1:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(progress 1.0 e 113/113 lb 1:ff7b1f36:::obj1:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ff7b1f36:::obj1:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 1.0 e 113/113 lb 1:ff7b1f36:::obj1:head) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 1.0 e 113/113 lb 1:ff7b1f36:::obj1:head) v3 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff38ecdd:::obj155:head local-lis/les=109/110 n=598 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 1.0 e 113/113 lb 1:ff7b1f36:::obj1:head) 2026-02-11T23:31:17.482+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 1.0 e 113/113 lb 1:ff7b1f36:::obj1:head) v3 finish latency 0.000022 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff7b1f36:::obj1:head local-lis/les=109/110 n=599 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 289 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3e4a000 2026-02-11T23:31:17.482+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 113 2026-02-11T23:31:17.482+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 290 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2e69a8fc:::obj598:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3e4a000 2026-02-11T23:31:17.482+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2e69a8fc:::obj598:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.482+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2e69a8fc:::obj598:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 113/112 [PushOp(4:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2e69a8fc:::obj598:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2e72dfbc:::obj232:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2e72dfbc:::obj232:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2e72dfbc:::obj232:head 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2e72dfbc:::obj232:head v 107'232 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.482+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.482+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2e72dfbc:::obj232:head, version: 107'232, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2e72dfbc:::obj232:head@107'232, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000038 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2e69a8fc:::obj598:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2e69a8fc:::obj598:head) v3 prio 15 cost 0 latency 0.000054 pg_backfill(progress 4.0 e 113/113 lb 4:2e69a8fc:::obj598:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2e69a8fc:::obj598:head) v3 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e5f3a17:::obj520:head local-lis/les=112/113 n=121 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2e69a8fc:::obj598:head) 2026-02-11T23:31:17.482+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2e69a8fc:::obj598:head) v3 finish latency 0.000012 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff7b1f36:::obj1:head local-lis/les=109/110 n=599 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.1 from_epoch 113 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.482+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_trim(1.0 107'600 e113/113) -- 0x4a04c00 con 0x3752000 2026-02-11T23:31:17.482+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- MOSDPGPushReply(1.0 113/109 [PushReplyOp(1:ffd8ed3d:::obj316:head)]) -- 0x4c81080 con 0x3752000 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.482+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.482+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7d080 con 0x3e4a000 2026-02-11T23:31:17.482+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2e72dfbc:::obj232:head)]) -- 0x4c81380 con 0x3e4a000 2026-02-11T23:31:17.482+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1239 ==== pg_backfill(finish 1.0 e 113/113 lb MAX) ==== 1098+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3752000 2026-02-11T23:31:17.483+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(finish 1.0 e 113/113 lb MAX) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 113 2026-02-11T23:31:17.483+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(finish 1.0 e 113/113 lb MAX) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(finish 1.0 e 113/113 lb MAX) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGRecoveryMsg(op=pg_backfill(finish 1.0 e 113/113 lb MAX) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4a2a000 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(finish 1.0 e 113/113 lb MAX) v3 prio 15 cost 0 latency 0.000021 pg_backfill(finish 1.0 e 113/113 lb MAX) v3 pg pg[1.0( v 107'600 (107'590,107'600] lb 1:ff7b1f36:::obj1:head local-lis/les=109/110 n=599 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.483+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.483+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff7b1f36:::obj1:head local-lis/les=109/110 n=599 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(finish 1.0 e 113/113 lb MAX) v3 2026-02-11T23:31:17.483+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] lb 1:ff7b1f36:::obj1:head local-lis/les=109/110 n=599 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(finish 1.0 e 113/113 lb MAX) 2026-02-11T23:31:17.483+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_backfill(finish_ack 1.0 e 113/113 lb MIN) -- 0x48bc100 con 0x3752000 2026-02-11T23:31:17.483+0000 7fac51168640 15 osd.0 113 enqueue_peering_evt 1.0 epoch_sent: 113 epoch_requested: 113 RecoveryDone 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RecoveryDone) class_id 2 prio 255 cost 10 e113) 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=599 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::update_backfill_progress final 2457600 replaces local 2457600 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RecoveryDone) class_id 2immediate prio 255 cost 10 e113) queued 2026-02-11T23:31:17.483+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.483+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(finish 1.0 e 113/113 lb MAX) v3 finish latency 0.000031 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RecoveryDone) class_id 2immediate prio 255 cost 10 e113) pg 0x4a2a000 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 113 epoch_requested: 113 RecoveryDone 2026-02-11T23:31:17.483+0000 7fac55170640 5 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] exit Started/ReplicaActive/RepRecovering 0.387029 10 0.000037 2026-02-11T23:31:17.483+0000 7fac55170640 5 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:17.483+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1240 ==== MBackfillReserve(1.0 RELEASE e113/113) ==== 46+0+0 (unknown 0 0 0) 0x4c81380 con 0x3752000 2026-02-11T23:31:17.483+0000 7fac703fa640 15 osd.0 113 enqueue_peering_evt 1.0 epoch_sent: 113 epoch_requested: 113 RemoteReservationCanceled 2026-02-11T23:31:17.483+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RemoteReservationCanceled) class_id 2 prio 255 cost 10 e113) 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RemoteReservationCanceled) class_id 2immediate prio 255 cost 10 e113) queued 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 RemoteReservationCanceled) class_id 2immediate prio 255 cost 10 e113) pg 0x4a2a000 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 113 epoch_requested: 113 RemoteReservationCanceled 2026-02-11T23:31:17.483+0000 7fac55170640 5 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.000108 1 0.000010 2026-02-11T23:31:17.483+0000 7fac55170640 5 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:17.483+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1241 ==== pg_info2(1.0 1.0 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) e113/113) ==== 1122+0+0 (unknown 0 0 0) 0x5175800 con 0x3752000 2026-02-11T23:31:17.483+0000 7fac703fa640 15 osd.0 113 enqueue_peering_evt 1.0 epoch_sent: 113 epoch_requested: 113 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) pg_lease(ru 83.949714661s ub 83.949714661s int 16.000000000s) 2026-02-11T23:31:17.483+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) pg_lease(ru 83.949714661s ub 83.949714661s int 16.000000000s)) class_id 2 prio 255 cost 10 e113) 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) pg_lease(ru 83.949714661s ub 83.949714661s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e113) queued 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 113 epoch_requested: 113 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) pg_lease(ru 83.949714661s ub 83.949714661s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e113) pg 0x4a2a000 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 113 epoch_requested: 113 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) pg_lease(ru 83.949714661s ub 83.949714661s int 16.000000000s) 2026-02-11T23:31:17.483+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 291 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4680 con 0x3e4a000 2026-02-11T23:31:17.483+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.483+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 292 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2e72dfbc:::obj232:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bc100 con 0x3e4a000 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.483+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2e72dfbc:::obj232:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 113 2026-02-11T23:31:17.483+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2e72dfbc:::obj232:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 113/112 [PushOp(4:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2e72dfbc:::obj232:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2eeda62f:::obj154:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2eeda62f:::obj154:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2eeda62f:::obj154:head 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2eeda62f:::obj154:head v 107'154 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.483+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2eeda62f:::obj154:head, version: 107'154, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2eeda62f:::obj154:head@107'154, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2e72dfbc:::obj232:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.483+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2e72dfbc:::obj232:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 113/113 lb 4:2e72dfbc:::obj232:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.483+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.483+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2e72dfbc:::obj232:head) v3 2026-02-11T23:31:17.483+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e69a8fc:::obj598:head local-lis/les=112/113 n=122 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2e72dfbc:::obj232:head) 2026-02-11T23:31:17.483+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.483+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2e72dfbc:::obj232:head) v3 finish latency 0.000013 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.483+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.483+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4eabc80 con 0x3e4a000 2026-02-11T23:31:17.483+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2eeda62f:::obj154:head)]) -- 0x4c97680 con 0x3e4a000 2026-02-11T23:31:17.483+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.484+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 293 ==== MOSDPGPush(4.0 113/112 [PushOp(4:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183040 con 0x3e4a000 2026-02-11T23:31:17.484+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.484+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.484+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 294 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2eeda62f:::obj154:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3e4a000 2026-02-11T23:31:17.484+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2eeda62f:::obj154:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.484+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2eeda62f:::obj154:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2eeda62f:::obj154:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:2f478301:::obj479:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:2f478301:::obj479:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:2f478301:::obj479:head 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:2f478301:::obj479:head v 107'479 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.484+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:2f478301:::obj479:head, version: 107'479, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:2f478301:::obj479:head@107'479, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2eeda62f:::obj154:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.484+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2eeda62f:::obj154:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 113/113 lb 4:2eeda62f:::obj154:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.484+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.484+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2eeda62f:::obj154:head) v3 2026-02-11T23:31:17.484+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2e72dfbc:::obj232:head local-lis/les=112/113 n=123 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2eeda62f:::obj154:head) 2026-02-11T23:31:17.484+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.484+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2eeda62f:::obj154:head) v3 finish latency 0.000012 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.484+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.484+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7ca80 con 0x3e4a000 2026-02-11T23:31:17.484+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:2f478301:::obj479:head)]) -- 0x4c81680 con 0x3e4a000 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.484+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 295 ==== MOSDPGPush(4.0 113/112 [PushOp(4:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.484+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.484+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.484+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 296 ==== pg_backfill(progress 4.0 e 113/113 lb 4:2f478301:::obj479:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.484+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:2f478301:::obj479:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.484+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2f478301:::obj479:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 113/112 [PushOp(4:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2f478301:::obj479:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:303427ce:::obj485:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:303427ce:::obj485:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:303427ce:::obj485:head 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:303427ce:::obj485:head v 107'485 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.484+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.484+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:303427ce:::obj485:head, version: 107'485, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:303427ce:::obj485:head@107'485, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.484+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:2f478301:::obj479:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.484+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2f478301:::obj479:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 113/113 lb 4:2f478301:::obj479:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.484+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.484+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:2f478301:::obj479:head) v3 2026-02-11T23:31:17.484+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2eeda62f:::obj154:head local-lis/les=112/113 n=124 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:2f478301:::obj479:head) 2026-02-11T23:31:17.484+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.484+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:2f478301:::obj479:head) v3 finish latency 0.000009 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.484+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.485+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.485+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.485+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c97800 con 0x3e4a000 2026-02-11T23:31:17.485+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:303427ce:::obj485:head)]) -- 0x4c81980 con 0x3e4a000 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.485+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 297 ==== MOSDPGPush(4.0 113/112 [PushOp(4:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.485+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.485+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.485+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.485+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 298 ==== pg_backfill(progress 4.0 e 113/113 lb 4:303427ce:::obj485:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3e4a000 2026-02-11T23:31:17.485+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:303427ce:::obj485:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.485+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.485+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:303427ce:::obj485:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.485+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.485+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.485+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 113/112 [PushOp(4:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.485+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:303427ce:::obj485:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.485+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.485+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.485+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:320b4068:::obj138:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.485+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:320b4068:::obj138:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.485+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:320b4068:::obj138:head 2026-02-11T23:31:17.485+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:320b4068:::obj138:head v 107'138 2026-02-11T23:31:17.485+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.485+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.485+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:320b4068:::obj138:head, version: 107'138, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:320b4068:::obj138:head@107'138, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.485+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:303427ce:::obj485:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.485+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:303427ce:::obj485:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 113/113 lb 4:303427ce:::obj485:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.485+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.485+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:303427ce:::obj485:head) v3 2026-02-11T23:31:17.485+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:2f478301:::obj479:head local-lis/les=112/113 n=125 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:303427ce:::obj485:head) 2026-02-11T23:31:17.485+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.485+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:303427ce:::obj485:head) v3 finish latency 0.000009 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.485+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.485+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.485+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7c900 con 0x3e4a000 2026-02-11T23:31:17.485+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:320b4068:::obj138:head)]) -- 0x4c81c80 con 0x3e4a000 2026-02-11T23:31:17.485+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.485+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 299 ==== MOSDPGPush(4.0 113/112 [PushOp(4:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.485+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.485+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.485+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.485+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.485+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 300 ==== pg_backfill(progress 4.0 e 113/113 lb 4:320b4068:::obj138:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.486+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:320b4068:::obj138:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 113 2026-02-11T23:31:17.486+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:320b4068:::obj138:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 113/112 [PushOp(4:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:320b4068:::obj138:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:325eabfb:::obj310:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:325eabfb:::obj310:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:325eabfb:::obj310:head 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:325eabfb:::obj310:head v 107'310 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.486+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:325eabfb:::obj310:head, version: 107'310, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:325eabfb:::obj310:head@107'310, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:320b4068:::obj138:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:320b4068:::obj138:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 113/113 lb 4:320b4068:::obj138:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:320b4068:::obj138:head) v3 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:303427ce:::obj485:head local-lis/les=112/113 n=126 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:320b4068:::obj138:head) 2026-02-11T23:31:17.486+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:320b4068:::obj138:head) v3 finish latency 0.000011 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.486+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c96000 con 0x3e4a000 2026-02-11T23:31:17.486+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:325eabfb:::obj310:head)]) -- 0x4eabc80 con 0x3e4a000 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.486+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 301 ==== MOSDPGPush(4.0 113/112 [PushOp(4:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.486+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.486+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.486+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 302 ==== pg_backfill(progress 4.0 e 113/113 lb 4:325eabfb:::obj310:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.486+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:325eabfb:::obj310:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.486+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:325eabfb:::obj310:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 113/112 [PushOp(4:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:325eabfb:::obj310:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3263b485:::obj202:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3263b485:::obj202:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3263b485:::obj202:head 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3263b485:::obj202:head v 107'202 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.486+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.486+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:3263b485:::obj202:head, version: 107'202, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3263b485:::obj202:head@107'202, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:325eabfb:::obj310:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:325eabfb:::obj310:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 113/113 lb 4:325eabfb:::obj310:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:325eabfb:::obj310:head) v3 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:320b4068:::obj138:head local-lis/les=112/113 n=127 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:325eabfb:::obj310:head) 2026-02-11T23:31:17.486+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:325eabfb:::obj310:head) v3 finish latency 0.000011 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.486+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.486+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.486+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7c780 con 0x3e4a000 2026-02-11T23:31:17.486+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:3263b485:::obj202:head)]) -- 0x4b7d080 con 0x3e4a000 2026-02-11T23:31:17.486+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.487+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 303 ==== MOSDPGPush(4.0 113/112 [PushOp(4:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3e4a000 2026-02-11T23:31:17.487+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.487+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.487+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 304 ==== pg_backfill(progress 4.0 e 113/113 lb 4:3263b485:::obj202:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.487+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:3263b485:::obj202:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 113 2026-02-11T23:31:17.487+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.487+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3263b485:::obj202:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.487+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.487+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.487+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.487+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 113/112 [PushOp(4:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.487+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3263b485:::obj202:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.487+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.487+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.487+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:326c82ab:::obj527:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.487+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:326c82ab:::obj527:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.487+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:326c82ab:::obj527:head 2026-02-11T23:31:17.487+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:326c82ab:::obj527:head v 107'527 2026-02-11T23:31:17.487+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.487+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.487+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:326c82ab:::obj527:head, version: 107'527, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:326c82ab:::obj527:head@107'527, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.487+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3263b485:::obj202:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.487+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:3263b485:::obj202:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 4.0 e 113/113 lb 4:3263b485:::obj202:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.487+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.487+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:3263b485:::obj202:head) v3 2026-02-11T23:31:17.487+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:325eabfb:::obj310:head local-lis/les=112/113 n=128 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:3263b485:::obj202:head) 2026-02-11T23:31:17.487+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.487+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:3263b485:::obj202:head) v3 finish latency 0.000011 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.487+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.487+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.487+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4fcfe00 con 0x3e4a000 2026-02-11T23:31:17.487+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:326c82ab:::obj527:head)]) -- 0x4a04c00 con 0x3e4a000 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.487+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 305 ==== MOSDPGPush(4.0 113/112 [PushOp(4:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3e4a000 2026-02-11T23:31:17.487+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.487+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.487+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 306 ==== pg_backfill(progress 4.0 e 113/113 lb 4:326c82ab:::obj527:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.487+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:326c82ab:::obj527:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 113 2026-02-11T23:31:17.487+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:326c82ab:::obj527:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.487+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.487+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 113/112 [PushOp(4:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:326c82ab:::obj527:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3286c571:::obj350:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3286c571:::obj350:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3286c571:::obj350:head 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3286c571:::obj350:head v 107'350 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.488+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:3286c571:::obj350:head, version: 107'350, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3286c571:::obj350:head@107'350, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:326c82ab:::obj527:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:326c82ab:::obj527:head) v3 prio 15 cost 0 latency 0.000104 pg_backfill(progress 4.0 e 113/113 lb 4:326c82ab:::obj527:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:326c82ab:::obj527:head) v3 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3263b485:::obj202:head local-lis/les=112/113 n=129 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:326c82ab:::obj527:head) 2026-02-11T23:31:17.488+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:326c82ab:::obj527:head) v3 finish latency 0.000017 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.488+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7c480 con 0x3e4a000 2026-02-11T23:31:17.488+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:3286c571:::obj350:head)]) -- 0x4b7c600 con 0x3e4a000 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.488+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 307 ==== MOSDPGPush(4.0 113/112 [PushOp(4:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.488+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 113 2026-02-11T23:31:17.488+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.488+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 308 ==== pg_backfill(progress 4.0 e 113/113 lb 4:3286c571:::obj350:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.488+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:3286c571:::obj350:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.488+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3286c571:::obj350:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 113/112 [PushOp(4:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3286c571:::obj350:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:32db648a:::obj568:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:32db648a:::obj568:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:32db648a:::obj568:head 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:32db648a:::obj568:head v 107'568 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.488+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.488+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:32db648a:::obj568:head, version: 107'568, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:32db648a:::obj568:head@107'568, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3286c571:::obj350:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:3286c571:::obj350:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 113/113 lb 4:3286c571:::obj350:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:3286c571:::obj350:head) v3 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:326c82ab:::obj527:head local-lis/les=112/113 n=130 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:3286c571:::obj350:head) 2026-02-11T23:31:17.488+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:3286c571:::obj350:head) v3 finish latency 0.000015 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.488+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.488+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.488+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c97e00 con 0x3e4a000 2026-02-11T23:31:17.488+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:32db648a:::obj568:head)]) -- 0x4b7d200 con 0x3e4a000 2026-02-11T23:31:17.488+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.489+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 309 ==== MOSDPGPush(4.0 113/112 [PushOp(4:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3e4a000 2026-02-11T23:31:17.489+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.489+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.489+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 310 ==== pg_backfill(progress 4.0 e 113/113 lb 4:32db648a:::obj568:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.489+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:32db648a:::obj568:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 113 2026-02-11T23:31:17.489+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:32db648a:::obj568:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 113/112 [PushOp(4:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:32db648a:::obj568:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3372ec01:::obj225:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3372ec01:::obj225:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3372ec01:::obj225:head 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3372ec01:::obj225:head v 107'225 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.489+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:3372ec01:::obj225:head, version: 107'225, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3372ec01:::obj225:head@107'225, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:32db648a:::obj568:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.489+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:32db648a:::obj568:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 113/113 lb 4:32db648a:::obj568:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.489+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.489+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:32db648a:::obj568:head) v3 2026-02-11T23:31:17.489+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3286c571:::obj350:head local-lis/les=112/113 n=131 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:32db648a:::obj568:head) 2026-02-11T23:31:17.489+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.489+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:32db648a:::obj568:head) v3 finish latency 0.000025 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.489+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.489+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4b7c300 con 0x3e4a000 2026-02-11T23:31:17.489+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:3372ec01:::obj225:head)]) -- 0x4b7cd80 con 0x3e4a000 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.489+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 311 ==== MOSDPGPush(4.0 113/112 [PushOp(4:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.489+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.489+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.489+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 312 ==== pg_backfill(progress 4.0 e 113/113 lb 4:3372ec01:::obj225:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.489+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:3372ec01:::obj225:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 113 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.489+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3372ec01:::obj225:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 113/112 [PushOp(4:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.489+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3372ec01:::obj225:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:339fd60b:::obj93:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.489+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:339fd60b:::obj93:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:339fd60b:::obj93:head 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:339fd60b:::obj93:head v 107'93 2026-02-11T23:31:17.489+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.489+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.490+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:339fd60b:::obj93:head, version: 107'93, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:339fd60b:::obj93:head@107'93, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.490+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3372ec01:::obj225:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:3372ec01:::obj225:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 113/113 lb 4:3372ec01:::obj225:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:3372ec01:::obj225:head) v3 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:32db648a:::obj568:head local-lis/les=112/113 n=132 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:3372ec01:::obj225:head) 2026-02-11T23:31:17.490+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:3372ec01:::obj225:head) v3 finish latency 0.000013 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.490+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.490+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4c97980 con 0x3e4a000 2026-02-11T23:31:17.490+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:339fd60b:::obj93:head)]) -- 0x4b7d380 con 0x3e4a000 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.490+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 313 ==== MOSDPGPush(4.0 113/112 [PushOp(4:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3e4a000 2026-02-11T23:31:17.490+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 113 2026-02-11T23:31:17.490+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.490+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 314 ==== pg_backfill(progress 4.0 e 113/113 lb 4:339fd60b:::obj93:head) ==== 1061+0+0 (unknown 0 0 0) 0x5165700 con 0x3e4a000 2026-02-11T23:31:17.490+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:339fd60b:::obj93:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 113 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.490+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:339fd60b:::obj93:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000050 MOSDPGPush(4.0 113/112 [PushOp(4:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.490+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.490+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:339fd60b:::obj93:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:33a4b6a3:::obj360:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:33a4b6a3:::obj360:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:33a4b6a3:::obj360:head 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:33a4b6a3:::obj360:head v 107'360 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.490+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:33a4b6a3:::obj360:head, version: 107'360, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:33a4b6a3:::obj360:head@107'360, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.490+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.490+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:339fd60b:::obj93:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.490+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:339fd60b:::obj93:head) v3 prio 15 cost 0 latency 0.000104 pg_backfill(progress 4.0 e 113/113 lb 4:339fd60b:::obj93:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.490+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.490+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:339fd60b:::obj93:head) v3 2026-02-11T23:31:17.490+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:3372ec01:::obj225:head local-lis/les=112/113 n=133 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:339fd60b:::obj93:head) 2026-02-11T23:31:17.490+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.490+0000 7fac51168640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:339fd60b:::obj93:head) v3 finish latency 0.000017 2026-02-11T23:31:17.490+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.490+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.490+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.490+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4e5c000 con 0x3e4a000 2026-02-11T23:31:17.490+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:33a4b6a3:::obj360:head)]) -- 0x4b7c180 con 0x3e4a000 2026-02-11T23:31:17.490+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.491+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 315 ==== MOSDPGPush(4.0 113/112 [PushOp(4:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3e4a000 2026-02-11T23:31:17.491+0000 7fac703fa640 15 osd.0 113 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000015 epoch 113 2026-02-11T23:31:17.491+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.491+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 316 ==== pg_backfill(progress 4.0 e 113/113 lb 4:33a4b6a3:::obj360:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165180 con 0x3e4a000 2026-02-11T23:31:17.491+0000 7fac703fa640 15 osd.0 113 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:33a4b6a3:::obj360:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 113 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.491+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:33a4b6a3:::obj360:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000045 MOSDPGPush(4.0 113/112 [PushOp(4:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:33a4b6a3:::obj360:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3421eea3:::obj166:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3421eea3:::obj166:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3421eea3:::obj166:head 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3421eea3:::obj166:head v 107'166 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.491+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 113 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:3421eea3:::obj166:head, version: 107'166, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3421eea3:::obj166:head@107'166, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:33a4b6a3:::obj360:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.491+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:33a4b6a3:::obj360:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 113/113 lb 4:33a4b6a3:::obj360:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.491+0000 7fac55170640 10 osd.0 113 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 113 is not newer than session's projected_epoch of 113 2026-02-11T23:31:17.491+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:33a4b6a3:::obj360:head) v3 2026-02-11T23:31:17.491+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:339fd60b:::obj93:head local-lis/les=112/113 n=134 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:33a4b6a3:::obj360:head) 2026-02-11T23:31:17.491+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.491+0000 7fac55170640 10 osd.0 113 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:33a4b6a3:::obj360:head) v3 finish latency 0.000022 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.491+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1242 ==== pg_query2(1.0 1.0 query(info 0'0 epoch_sent 114) e114/114) ==== 158+0+0 (unknown 0 0 0) 0x4c82d20 con 0x3752000 2026-02-11T23:31:17.491+0000 7fac703fa640 15 osd.0 113 enqueue_peering_evt 1.0 epoch_sent: 114 epoch_requested: 114 MQuery 1.0 from 1 query_epoch 114 query: query(info 0'0 epoch_sent 114) 2026-02-11T23:31:17.491+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 MQuery 1.0 from 1 query_epoch 114 query: query(info 0'0 epoch_sent 114)) class_id 2 prio 255 cost 10 e114) 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 MQuery 1.0 from 1 query_epoch 114 query: query(info 0'0 epoch_sent 114)) class_id 2immediate prio 255 cost 10 e114) queued 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 MQuery 1.0 from 1 query_epoch 114 query: query(info 0'0 epoch_sent 114)) class_id 2immediate prio 255 cost 10 e114) pg 0x4a2a000 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 114, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 MQuery 1.0 from 1 query_epoch 114 query: query(info 0'0 epoch_sent 114)) class_id 2immediate prio 255 cost 10 e114) 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.491+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1243 ==== osd_map(114..114 src has 1..114) ==== 317+0+0 (unknown 0 0 0) 0x46eaa80 con 0x3752000 2026-02-11T23:31:17.491+0000 7fac606e4640 20 osd.0 113 OSD::ms_dispatch: osd_map(114..114 src has 1..114) 2026-02-11T23:31:17.491+0000 7fac606e4640 20 osd.0 113 _dispatch 0x46eaa80 osd_map(114..114 src has 1..114) 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.491+0000 7fac606e4640 3 osd.0 113 handle_osd_map epochs [114,114], i have 113, src has [1,114] 2026-02-11T23:31:17.491+0000 7fac606e4640 10 osd.0 113 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.491+0000 7fac606e4640 10 osd.0 113 handle_osd_map got inc map for epoch 114 2026-02-11T23:31:17.491+0000 7fac606e4640 20 osd.0 113 got_full_map 114, nothing requested 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.491+0000 7fac606e4640 20 osd.0 113 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:17.491+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 113 get_con_osd_cluster to osd.4 from_epoch 113 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 113 get_nextmap_reserved map_reservations: {113=1} 2026-02-11T23:31:17.491+0000 7fac606e4640 20 osd.0 113 handle_osd_map pg_num_history pg_num_history(e114 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 113 release_map epoch: 113 2026-02-11T23:31:17.491+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {114={}} 2026-02-11T23:31:17.491+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:17.491+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e113/113) -- 0x4e5c180 con 0x3e4a000 2026-02-11T23:31:17.491+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e114 maps [1~114] lci=[5,114] tlb=1) 2026-02-11T23:31:17.491+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:3421eea3:::obj166:head)]) -- 0x4c80600 con 0x3e4a000 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.491+0000 7fac5e983640 10 osd.0 113 _committed_osd_maps 114..114 2026-02-11T23:31:17.491+0000 7fac5e983640 10 osd.0 113 advance to epoch 114 (<= last 114 <= newest_map 114) 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0 114 consume_map version 114 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0 113 await_reserved_maps epoch:114 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0 113 await_reserved_maps done 0 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0 114 consume_map 114 -- publish done 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x4946160 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0 114 identify_splits_and_merges 4.0 e113 to e114 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0 114 identify_splits_and_merges 4.0 e113 to e114 pg_nums {45=1} 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x25809a0 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0 114 identify_splits_and_merges 1.0 e113 to e114 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0 114 identify_splits_and_merges 1.0 e113 to e114 pg_nums {24=1} 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0 114 prune_pg_created 2026-02-11T23:31:17.491+0000 7fac5e983640 10 osd.0:0.consume_map 114 (was 113) 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 114 <= 114, requeueing 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {114=} 2026-02-11T23:31:17.491+0000 7fac5e983640 10 osd.0:1.consume_map 114 (was 113) 2026-02-11T23:31:17.491+0000 7fac5e983640 10 osd.0:2.consume_map 114 (was 113) 2026-02-11T23:31:17.491+0000 7fac5e983640 10 osd.0:3.consume_map 114 (was 113) 2026-02-11T23:31:17.491+0000 7fac5e983640 10 osd.0:4.consume_map 114 (was 113) 2026-02-11T23:31:17.491+0000 7fac5e983640 10 osd.0:5.consume_map 114 (was 113) 2026-02-11T23:31:17.491+0000 7fac5e983640 10 osd.0:6.consume_map 114 (was 113) 2026-02-11T23:31:17.491+0000 7fac5e983640 10 osd.0:7.consume_map 114 (was 113) 2026-02-11T23:31:17.491+0000 7fac5e983640 10 osd.0 114 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.491+0000 7fac5e983640 15 osd.0 114 enqueue_peering_evt 4.0 epoch_sent: 114 epoch_requested: 114 NullEvt 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 MQuery 1.0 from 1 query_epoch 114 query: query(info 0'0 epoch_sent 114)) class_id 2immediate prio 255 cost 10 e114) queued 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 NullEvt) class_id 2 prio 255 cost 10 e114) 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 MQuery 1.0 from 1 query_epoch 114 query: query(info 0'0 epoch_sent 114)) class_id 2immediate prio 255 cost 10 e114) pg 0x4a2a000 2026-02-11T23:31:17.491+0000 7fac5e983640 15 osd.0 114 enqueue_peering_evt 1.0 epoch_sent: 114 epoch_requested: 114 NullEvt 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.491+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 NullEvt) class_id 2 prio 255 cost 10 e114) 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 NullEvt) class_id 2immediate prio 255 cost 10 e114) queued 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] handle_advance_map: 114 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 113 -> 114 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 NullEvt) class_id 2immediate prio 255 cost 10 e114) pg 0x4859000 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 pg_epoch: 113 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:31:17.491+0000 7fac5e983640 7 osd.0 114 activate_map version 114 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] state: Started advmap 2026-02-11T23:31:17.491+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::should_restart_peering new interval newup [1,0] newacting [1,0] 2026-02-11T23:31:17.491+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_advance_map: 114 2026-02-11T23:31:17.491+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:31:17.491+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 113 -> 114 2026-02-11T23:31:17.491+0000 7fac55971640 20 osd.0 114 agent_entry empty queue 2026-02-11T23:31:17.491+0000 7fac51168640 5 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.008787 2 0.000005 2026-02-11T23:31:17.491+0000 7fac55170640 10 osd.0 pg_epoch: 113 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4] -- 4/4 2026-02-11T23:31:17.491+0000 7fac51168640 5 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] exit Started/ReplicaActive 0.398418 0 0.000000 2026-02-11T23:31:17.491+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] state: Started advmap 2026-02-11T23:31:17.491+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.491+0000 7fac51168640 5 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] exit Started 0.398681 0 0.000000 2026-02-11T23:31:17.492+0000 7fac51168640 5 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] enter Reset 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 114 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=109 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 114 2026-02-11T23:31:17.492+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_activate_map: epoch range: 114..114 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=114 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:31:17.492+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 113 while current is 114 2026-02-11T23:31:17.492+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=114 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:31:17.492+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 114 epoch_requested: 114 NullEvt 2026-02-11T23:31:17.492+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=114 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] state: Reset advmap 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 NullEvt) class_id 2immediate prio 255 cost 10 e114) queued 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=114 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::should_restart_peering new interval newup [1,0] newacting [1,0] 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=114 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0]/[1] r=-1 lpr=114 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 114 2026-02-11T23:31:17.492+0000 7fac51168640 15 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0] r=-1 lpr=114 pi=[103,109)/1 pct=0'0 crt=107'600 active mbc={}] clear_stats 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(109-113 up [1, 0](1) acting [1](1)) up_thru 109 up_from 9 last_epoch_clean 105 interval(109-113 up [1, 0](1) acting [1](1) maybe_went_rw) : primary up 9-109 includes interval 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=109) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::start_peering_interval noting past ([103,113] all_participants={0, 1} intervals=[([109,113] acting={1})]) 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [1,0]+[1,0] 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::init_hb_stamps now [0x3533ab0] 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 70.326858521s + 0.000000000s) 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] cancel_recovery 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] clear_recovery_state 2026-02-11T23:31:17.492+0000 7fac51168640 15 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] _clear_recovery_state 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:17.492+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:31:17.492+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:17.492+0000 7fac51168640 1 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::start_peering_interval up [1,0] -> [1,0], acting [1] -> [1,0], acting_primary 1 -> 1, up_primary 1 -> 1, role -1 -> 1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] clear_primary_state 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] release_backoffs [1:00000000::::0,MAX) 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:17.492+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:17.492+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] agent_stop 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] on_change 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] cancel_copy_ops 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] cancel_flush_ops 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] cancel_proxy_ops 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] cancel_manifest_ops 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] cancel_cls_gather_ops 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] on_change_cleanup 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] on_change 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] exit NotTrimming 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] enter NotTrimming 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] plpg_on_role_change 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 114 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] handle_activate_map: epoch range: 114..114 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.492+0000 7fac51168640 5 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] exit Reset 0.000094 1 0.000104 2026-02-11T23:31:17.492+0000 7fac51168640 5 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] enter Started 2026-02-11T23:31:17.492+0000 7fac51168640 5 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] enter Start 2026-02-11T23:31:17.492+0000 7fac51168640 1 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] state: transitioning to Stray 2026-02-11T23:31:17.492+0000 7fac51168640 5 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] exit Start 0.000017 0 0.000000 2026-02-11T23:31:17.492+0000 7fac51168640 5 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] enter Started/Stray 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 113 while current is 114 2026-02-11T23:31:17.492+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 114 epoch_requested: 114 MQuery 1.0 from 1 query_epoch 114 query: query(info 0'0 epoch_sent 114) 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:31:17.492+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 114 get_con_osd_cluster to osd.1 from_epoch 114 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 113 -> 114 (shared) 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 114 send_incremental_map epoch range: (114, 114] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 114 add_map_inc_bl 114 273 bytes 2026-02-11T23:31:17.492+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(114..114 src has 1..114) -- 0x3941880 con 0x3752000 2026-02-11T23:31:17.492+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:114 sent:114 1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) ([103,113] all_participants={0, 1} intervals=[([109,113] acting={1})])) e114/114) -- 0x4e5a300 con 0x3752000 2026-02-11T23:31:17.492+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:114 sent:114 1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) ([103,113] all_participants={0, 1} intervals=[([109,113] acting={1})])) e114/114) -- 0x4e5aa00 con 0x3752000 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 114 epoch_requested: 114 NullEvt) class_id 2immediate prio 255 cost 10 e114) pg 0x4a2a000 2026-02-11T23:31:17.492+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 114 epoch_requested: 114 NullEvt 2026-02-11T23:31:17.492+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.492+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 317 ==== MOSDPGPush(4.0 113/112 [PushOp(4:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.492+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 113 2026-02-11T23:31:17.492+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.492+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 318 ==== pg_backfill(progress 4.0 e 113/113 lb 4:3421eea3:::obj166:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164c00 con 0x3e4a000 2026-02-11T23:31:17.492+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:3421eea3:::obj166:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 113 2026-02-11T23:31:17.492+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3421eea3:::obj166:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 113/112 [PushOp(4:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 113 -> 114 (shared) 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3421eea3:::obj166:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 114 send_incremental_map epoch range: (114, 114] to v2:127.0.0.1:6836/864727417 2026-02-11T23:31:17.492+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(114..114 src has 1..114) -- 0x3940700 con 0x3e4a000 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:360ee05a:::obj519:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:360ee05a:::obj519:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:360ee05a:::obj519:head 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:360ee05a:::obj519:head v 107'519 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.492+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.492+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:360ee05a:::obj519:head, version: 107'519, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:360ee05a:::obj519:head@107'519, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000074 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:3421eea3:::obj166:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.492+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:3421eea3:::obj166:head) v3 prio 15 cost 0 latency 0.000105 pg_backfill(progress 4.0 e 113/113 lb 4:3421eea3:::obj166:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.492+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.492+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:3421eea3:::obj166:head) v3 2026-02-11T23:31:17.492+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:33a4b6a3:::obj360:head local-lis/les=112/113 n=135 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:3421eea3:::obj166:head) 2026-02-11T23:31:17.492+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.492+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:3421eea3:::obj166:head) v3 finish latency 0.000026 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.492+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.492+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5c300 con 0x3e4a000 2026-02-11T23:31:17.492+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:360ee05a:::obj519:head)]) -- 0x4b7d500 con 0x3e4a000 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.492+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.492+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.493+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 319 ==== MOSDPGPush(4.0 113/112 [PushOp(4:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.493+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 113/112 [PushOp(4:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 113 2026-02-11T23:31:17.493+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e113) 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) queued 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 113/112 [PushOp(4:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e113) pg 0x4859000 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 113/112 [PushOp(4:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 113/112 [PushOp(4:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3655ac21:::obj70:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3655ac21:::obj70:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3655ac21:::obj70:head 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3655ac21:::obj70:head v 107'70 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.493+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.493+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 320 ==== osd_map(114..114 src has 1..114) ==== 317+0+0 (unknown 0 0 0) 0x3940700 con 0x3e4a000 2026-02-11T23:31:17.493+0000 7fac606e4640 20 osd.0 114 OSD::ms_dispatch: osd_map(114..114 src has 1..114) 2026-02-11T23:31:17.493+0000 7fac606e4640 20 osd.0 114 _dispatch 0x3940700 osd_map(114..114 src has 1..114) 2026-02-11T23:31:17.493+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 321 ==== pg_backfill(progress 4.0 e 113/113 lb 4:360ee05a:::obj519:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3e4a000 2026-02-11T23:31:17.493+0000 7fac606e4640 3 osd.0 114 handle_osd_map epochs [114,114], i have 114, src has [1,114] 2026-02-11T23:31:17.493+0000 7fac606e4640 10 osd.0 114 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.493+0000 7fac606e4640 10 osd.0 114 no new maps here, dropping 2026-02-11T23:31:17.493+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 113/113 lb 4:360ee05a:::obj519:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 113 2026-02-11T23:31:17.493+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:360ee05a:::obj519:head) v3) class_id 0 prio 15 cost 0 e113) 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:360ee05a:::obj519:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) queued 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 113/112 [PushOp(4:3655ac21:::obj70:head, version: 107'70, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3655ac21:::obj70:head@107'70, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 113/113 lb 4:360ee05a:::obj519:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e113) pg 0x4859000 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:360ee05a:::obj519:head) v3 prio 15 cost 0 latency 0.000036 pg_backfill(progress 4.0 e 113/113 lb 4:360ee05a:::obj519:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 113/113 lb 4:360ee05a:::obj519:head) v3 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3421eea3:::obj166:head local-lis/les=112/113 n=136 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 113/113 lb 4:360ee05a:::obj519:head) 2026-02-11T23:31:17.493+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 113/113 lb 4:360ee05a:::obj519:head) v3 finish latency 0.000025 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.493+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5c600 con 0x3e4a000 2026-02-11T23:31:17.493+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 113/112 [PushReplyOp(4:3655ac21:::obj70:head)]) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.493+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 322 ==== MOSDPGPush(4.0 114/112 [PushOp(4:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.493+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.493+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.493+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 323 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3655ac21:::obj70:head) ==== 1061+0+0 (unknown 0 0 0) 0x4def180 con 0x3e4a000 2026-02-11T23:31:17.493+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3655ac21:::obj70:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 114 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.493+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3655ac21:::obj70:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 114/112 [PushOp(4:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3655ac21:::obj70:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:372fd24d:::obj442:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:372fd24d:::obj442:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:372fd24d:::obj442:head 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:372fd24d:::obj442:head v 107'442 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.493+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.493+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:372fd24d:::obj442:head, version: 107'442, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:372fd24d:::obj442:head@107'442, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3655ac21:::obj70:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3655ac21:::obj70:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 114/114 lb 4:3655ac21:::obj70:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3655ac21:::obj70:head) v3 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:360ee05a:::obj519:head local-lis/les=112/113 n=137 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3655ac21:::obj70:head) 2026-02-11T23:31:17.493+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3655ac21:::obj70:head) v3 finish latency 0.000015 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.493+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.493+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.493+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5c780 con 0x3e4a000 2026-02-11T23:31:17.493+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:372fd24d:::obj442:head)]) -- 0x4fcf200 con 0x3e4a000 2026-02-11T23:31:17.493+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.494+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 324 ==== MOSDPGPush(4.0 114/112 [PushOp(4:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.494+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.494+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.494+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.494+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.494+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 325 ==== pg_backfill(progress 4.0 e 114/114 lb 4:372fd24d:::obj442:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd180 con 0x3e4a000 2026-02-11T23:31:17.494+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.494+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:372fd24d:::obj442:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 114 2026-02-11T23:31:17.494+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:372fd24d:::obj442:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.494+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.494+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.494+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 114/112 [PushOp(4:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.494+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.494+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.494+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.494+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.494+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:372fd24d:::obj442:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.494+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:37ff9db7:::obj453:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.494+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:37ff9db7:::obj453:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.494+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:37ff9db7:::obj453:head 2026-02-11T23:31:17.494+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:37ff9db7:::obj453:head v 107'453 2026-02-11T23:31:17.494+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.494+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.494+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:37ff9db7:::obj453:head, version: 107'453, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:37ff9db7:::obj453:head@107'453, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.494+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.494+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.494+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:372fd24d:::obj442:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.494+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:372fd24d:::obj442:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 114/114 lb 4:372fd24d:::obj442:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.494+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.494+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:372fd24d:::obj442:head) v3 2026-02-11T23:31:17.494+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3655ac21:::obj70:head local-lis/les=112/113 n=138 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:372fd24d:::obj442:head) 2026-02-11T23:31:17.494+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.494+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:372fd24d:::obj442:head) v3 finish latency 0.000014 2026-02-11T23:31:17.494+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.494+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.494+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.494+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.494+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.494+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.494+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.494+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5c900 con 0x3e4a000 2026-02-11T23:31:17.494+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:37ff9db7:::obj453:head)]) -- 0x4b7d680 con 0x3e4a000 2026-02-11T23:31:17.494+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.495+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 326 ==== MOSDPGPush(4.0 114/112 [PushOp(4:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3e4a000 2026-02-11T23:31:17.495+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 114 2026-02-11T23:31:17.495+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.495+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 327 ==== pg_backfill(progress 4.0 e 114/114 lb 4:37ff9db7:::obj453:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3e4a000 2026-02-11T23:31:17.495+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:37ff9db7:::obj453:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.495+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:37ff9db7:::obj453:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 114/112 [PushOp(4:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:37ff9db7:::obj453:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:384eca0c:::obj484:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:384eca0c:::obj484:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:384eca0c:::obj484:head 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:384eca0c:::obj484:head v 107'484 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.495+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:384eca0c:::obj484:head, version: 107'484, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:384eca0c:::obj484:head@107'484, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:37ff9db7:::obj453:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.495+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:37ff9db7:::obj453:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 114/114 lb 4:37ff9db7:::obj453:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.495+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.495+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:37ff9db7:::obj453:head) v3 2026-02-11T23:31:17.495+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:372fd24d:::obj442:head local-lis/les=112/113 n=139 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:37ff9db7:::obj453:head) 2026-02-11T23:31:17.495+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.495+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:37ff9db7:::obj453:head) v3 finish latency 0.000011 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.495+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.495+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5ca80 con 0x3e4a000 2026-02-11T23:31:17.495+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:384eca0c:::obj484:head)]) -- 0x49b8f00 con 0x3e4a000 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.495+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 328 ==== MOSDPGPush(4.0 114/112 [PushOp(4:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51831e0 con 0x3e4a000 2026-02-11T23:31:17.495+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.495+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.495+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 329 ==== pg_backfill(progress 4.0 e 114/114 lb 4:384eca0c:::obj484:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3e4a000 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.495+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:384eca0c:::obj484:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 114 2026-02-11T23:31:17.495+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:384eca0c:::obj484:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 114/112 [PushOp(4:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:384eca0c:::obj484:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:385733f0:::obj251:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:385733f0:::obj251:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:385733f0:::obj251:head 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:385733f0:::obj251:head v 107'251 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.495+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.495+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:385733f0:::obj251:head, version: 107'251, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:385733f0:::obj251:head@107'251, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.495+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:384eca0c:::obj484:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.495+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:384eca0c:::obj484:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 114/114 lb 4:384eca0c:::obj484:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.495+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.495+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:384eca0c:::obj484:head) v3 2026-02-11T23:31:17.495+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:37ff9db7:::obj453:head local-lis/les=112/113 n=140 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:384eca0c:::obj484:head) 2026-02-11T23:31:17.495+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.495+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:384eca0c:::obj484:head) v3 finish latency 0.000011 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.495+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.496+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.496+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.496+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5cc00 con 0x3e4a000 2026-02-11T23:31:17.496+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:385733f0:::obj251:head)]) -- 0x4b7d800 con 0x3e4a000 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.496+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 330 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3e4a000 2026-02-11T23:31:17.496+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.496+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.496+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.496+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 331 ==== pg_backfill(progress 4.0 e 114/114 lb 4:385733f0:::obj251:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3e4a000 2026-02-11T23:31:17.496+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.496+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:385733f0:::obj251:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 114 2026-02-11T23:31:17.496+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:385733f0:::obj251:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.496+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.496+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.496+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 114/112 [PushOp(4:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.496+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:385733f0:::obj251:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.496+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.496+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.496+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3945af93:::obj271:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.496+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3945af93:::obj271:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.496+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3945af93:::obj271:head 2026-02-11T23:31:17.496+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3945af93:::obj271:head v 107'271 2026-02-11T23:31:17.496+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.496+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.496+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3945af93:::obj271:head, version: 107'271, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3945af93:::obj271:head@107'271, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.496+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:385733f0:::obj251:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.496+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:385733f0:::obj251:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 114/114 lb 4:385733f0:::obj251:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.496+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.496+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:385733f0:::obj251:head) v3 2026-02-11T23:31:17.496+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:384eca0c:::obj484:head local-lis/les=112/113 n=141 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:385733f0:::obj251:head) 2026-02-11T23:31:17.496+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.496+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:385733f0:::obj251:head) v3 finish latency 0.000011 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.496+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.496+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.496+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5cd80 con 0x3e4a000 2026-02-11T23:31:17.496+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3945af93:::obj271:head)]) -- 0x4c97200 con 0x3e4a000 2026-02-11T23:31:17.496+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.497+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 332 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51821a0 con 0x3e4a000 2026-02-11T23:31:17.497+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.497+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.497+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 333 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3945af93:::obj271:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3e4a000 2026-02-11T23:31:17.497+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3945af93:::obj271:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.497+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3945af93:::obj271:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 114/112 [PushOp(4:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3945af93:::obj271:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3a764ab4:::obj37:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3a764ab4:::obj37:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3a764ab4:::obj37:head 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3a764ab4:::obj37:head v 107'37 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.497+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3a764ab4:::obj37:head, version: 107'37, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3a764ab4:::obj37:head@107'37, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3945af93:::obj271:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.497+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3945af93:::obj271:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 114/114 lb 4:3945af93:::obj271:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.497+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.497+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3945af93:::obj271:head) v3 2026-02-11T23:31:17.497+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:385733f0:::obj251:head local-lis/les=112/113 n=142 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3945af93:::obj271:head) 2026-02-11T23:31:17.497+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.497+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3945af93:::obj271:head) v3 finish latency 0.000012 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.497+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.497+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5cf00 con 0x3e4a000 2026-02-11T23:31:17.497+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3a764ab4:::obj37:head)]) -- 0x4b7dc80 con 0x3e4a000 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.497+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 334 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3e4a000 2026-02-11T23:31:17.497+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.497+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.497+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 335 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3a764ab4:::obj37:head) ==== 1061+0+0 (unknown 0 0 0) 0x4ded080 con 0x3e4a000 2026-02-11T23:31:17.497+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3a764ab4:::obj37:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.497+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3a764ab4:::obj37:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 114/112 [PushOp(4:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3a764ab4:::obj37:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3b0aa4c9:::obj511:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3b0aa4c9:::obj511:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3b0aa4c9:::obj511:head 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3b0aa4c9:::obj511:head v 107'511 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.497+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.497+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3b0aa4c9:::obj511:head, version: 107'511, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3b0aa4c9:::obj511:head@107'511, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.497+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3a764ab4:::obj37:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.497+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3a764ab4:::obj37:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 114/114 lb 4:3a764ab4:::obj37:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.497+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.497+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3a764ab4:::obj37:head) v3 2026-02-11T23:31:17.497+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3945af93:::obj271:head local-lis/les=112/113 n=143 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3a764ab4:::obj37:head) 2026-02-11T23:31:17.497+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.497+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3a764ab4:::obj37:head) v3 finish latency 0.000013 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.497+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.498+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.498+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.498+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d080 con 0x3e4a000 2026-02-11T23:31:17.498+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3b0aa4c9:::obj511:head)]) -- 0x49b8c00 con 0x3e4a000 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.498+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 336 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3e4a000 2026-02-11T23:31:17.498+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 114 2026-02-11T23:31:17.498+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.498+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.498+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 337 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3b0aa4c9:::obj511:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3e4a000 2026-02-11T23:31:17.498+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3b0aa4c9:::obj511:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.498+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.498+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3b0aa4c9:::obj511:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.498+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.498+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.498+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 114/112 [PushOp(4:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.498+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3b0aa4c9:::obj511:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.498+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.498+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.498+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3bc05e5f:::obj343:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.498+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3bc05e5f:::obj343:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.498+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3bc05e5f:::obj343:head 2026-02-11T23:31:17.498+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3bc05e5f:::obj343:head v 107'343 2026-02-11T23:31:17.498+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.498+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.498+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3bc05e5f:::obj343:head, version: 107'343, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3bc05e5f:::obj343:head@107'343, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.498+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3b0aa4c9:::obj511:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.498+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3b0aa4c9:::obj511:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 114/114 lb 4:3b0aa4c9:::obj511:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.498+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.498+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3b0aa4c9:::obj511:head) v3 2026-02-11T23:31:17.498+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3a764ab4:::obj37:head local-lis/les=112/113 n=144 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3b0aa4c9:::obj511:head) 2026-02-11T23:31:17.498+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.498+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3b0aa4c9:::obj511:head) v3 finish latency 0.000013 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.498+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.498+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.498+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.498+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3bc05e5f:::obj343:head)]) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.498+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.499+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 338 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.499+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.499+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.499+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 339 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3bc05e5f:::obj343:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3e4a000 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.499+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3bc05e5f:::obj343:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 114 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.499+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3bc05e5f:::obj343:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 114/112 [PushOp(4:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3bc05e5f:::obj343:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3cb06661:::obj109:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3cb06661:::obj109:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3cb06661:::obj109:head 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3cb06661:::obj109:head v 107'109 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.499+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3cb06661:::obj109:head, version: 107'109, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb06661:::obj109:head@107'109, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3bc05e5f:::obj343:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3bc05e5f:::obj343:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 114/114 lb 4:3bc05e5f:::obj343:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3bc05e5f:::obj343:head) v3 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3b0aa4c9:::obj511:head local-lis/les=112/113 n=145 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3bc05e5f:::obj343:head) 2026-02-11T23:31:17.499+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3bc05e5f:::obj343:head) v3 finish latency 0.000014 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.499+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d380 con 0x3e4a000 2026-02-11T23:31:17.499+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3cb06661:::obj109:head)]) -- 0x49d7800 con 0x3e4a000 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.499+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 340 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.499+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.499+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.499+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 341 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3cb06661:::obj109:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3e4a000 2026-02-11T23:31:17.499+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3cb06661:::obj109:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 114 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.499+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3cb06661:::obj109:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 114/112 [PushOp(4:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3cb06661:::obj109:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3cb2c87d:::obj96:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3cb2c87d:::obj96:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3cb2c87d:::obj96:head 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3cb2c87d:::obj96:head v 107'96 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.499+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.499+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3cb2c87d:::obj96:head, version: 107'96, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cb2c87d:::obj96:head@107'96, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.499+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3cb06661:::obj109:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3cb06661:::obj109:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 4.0 e 114/114 lb 4:3cb06661:::obj109:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3cb06661:::obj109:head) v3 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3bc05e5f:::obj343:head local-lis/les=112/113 n=146 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3cb06661:::obj109:head) 2026-02-11T23:31:17.499+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.499+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3cb06661:::obj109:head) v3 finish latency 0.000019 2026-02-11T23:31:17.499+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.500+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.500+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.500+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d680 con 0x3e4a000 2026-02-11T23:31:17.500+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3cb2c87d:::obj96:head)]) -- 0x4e5d500 con 0x3e4a000 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.500+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 342 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.500+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.500+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.500+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.500+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 343 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3cb2c87d:::obj96:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3e4a000 2026-02-11T23:31:17.500+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3cb2c87d:::obj96:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.500+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.500+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3cb2c87d:::obj96:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.500+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.500+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.500+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 114/112 [PushOp(4:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.500+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3cb2c87d:::obj96:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.500+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.500+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.500+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3cc20580:::obj177:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.500+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3cc20580:::obj177:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.500+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3cc20580:::obj177:head 2026-02-11T23:31:17.500+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3cc20580:::obj177:head v 107'177 2026-02-11T23:31:17.500+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.500+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.500+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3cc20580:::obj177:head, version: 107'177, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3cc20580:::obj177:head@107'177, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.500+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3cb2c87d:::obj96:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.500+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3cb2c87d:::obj96:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 114/114 lb 4:3cb2c87d:::obj96:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.500+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.500+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3cb2c87d:::obj96:head) v3 2026-02-11T23:31:17.500+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb06661:::obj109:head local-lis/les=112/113 n=147 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3cb2c87d:::obj96:head) 2026-02-11T23:31:17.500+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.500+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3cb2c87d:::obj96:head) v3 finish latency 0.000019 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.500+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.500+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.500+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d800 con 0x3e4a000 2026-02-11T23:31:17.500+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3cc20580:::obj177:head)]) -- 0x4fcf200 con 0x3e4a000 2026-02-11T23:31:17.500+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.501+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 344 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.501+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.501+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.501+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 345 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3cc20580:::obj177:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3e4a000 2026-02-11T23:31:17.501+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3cc20580:::obj177:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 114 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.501+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3cc20580:::obj177:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 114/112 [PushOp(4:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3cc20580:::obj177:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3d7e2b4d:::obj143:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3d7e2b4d:::obj143:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3d7e2b4d:::obj143:head 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3d7e2b4d:::obj143:head v 107'143 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.501+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3d7e2b4d:::obj143:head, version: 107'143, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d7e2b4d:::obj143:head@107'143, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3cc20580:::obj177:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.501+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3cc20580:::obj177:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 114/114 lb 4:3cc20580:::obj177:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.501+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.501+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3cc20580:::obj177:head) v3 2026-02-11T23:31:17.501+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cb2c87d:::obj96:head local-lis/les=112/113 n=148 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3cc20580:::obj177:head) 2026-02-11T23:31:17.501+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.501+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3cc20580:::obj177:head) v3 finish latency 0.000013 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.501+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.501+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d980 con 0x3e4a000 2026-02-11T23:31:17.501+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3d7e2b4d:::obj143:head)]) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.501+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 346 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3e4a000 2026-02-11T23:31:17.501+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.501+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.501+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 347 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3d7e2b4d:::obj143:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3e4a000 2026-02-11T23:31:17.501+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3d7e2b4d:::obj143:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.501+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3d7e2b4d:::obj143:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 114/112 [PushOp(4:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3d7e2b4d:::obj143:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3d8750d8:::obj471:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3d8750d8:::obj471:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3d8750d8:::obj471:head 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3d8750d8:::obj471:head v 107'471 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.501+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.501+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3d8750d8:::obj471:head, version: 107'471, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d8750d8:::obj471:head@107'471, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.501+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3d7e2b4d:::obj143:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.501+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3d7e2b4d:::obj143:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 114/114 lb 4:3d7e2b4d:::obj143:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.501+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.501+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3d7e2b4d:::obj143:head) v3 2026-02-11T23:31:17.501+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3cc20580:::obj177:head local-lis/les=112/113 n=149 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3d7e2b4d:::obj143:head) 2026-02-11T23:31:17.501+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.501+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3d7e2b4d:::obj143:head) v3 finish latency 0.000013 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.501+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.502+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.502+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.502+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5db00 con 0x3e4a000 2026-02-11T23:31:17.502+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3d8750d8:::obj471:head)]) -- 0x4b7d500 con 0x3e4a000 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.502+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 348 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3e4a000 2026-02-11T23:31:17.502+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 114 2026-02-11T23:31:17.502+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.502+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 349 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3d8750d8:::obj471:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3e4a000 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.502+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3d8750d8:::obj471:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 114 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.502+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3d8750d8:::obj471:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.502+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.502+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 114/112 [PushOp(4:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.502+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.502+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.502+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.502+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.502+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3d8750d8:::obj471:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.502+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3d88a4f9:::obj551:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3d88a4f9:::obj551:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.502+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3d88a4f9:::obj551:head 2026-02-11T23:31:17.502+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3d88a4f9:::obj551:head v 107'551 2026-02-11T23:31:17.502+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.502+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.502+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3d88a4f9:::obj551:head, version: 107'551, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3d88a4f9:::obj551:head@107'551, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.502+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.502+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3d8750d8:::obj471:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.502+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3d8750d8:::obj471:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 114/114 lb 4:3d8750d8:::obj471:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.502+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.502+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3d8750d8:::obj471:head) v3 2026-02-11T23:31:17.502+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d7e2b4d:::obj143:head local-lis/les=112/113 n=150 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3d8750d8:::obj471:head) 2026-02-11T23:31:17.502+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.502+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3d8750d8:::obj471:head) v3 finish latency 0.000020 2026-02-11T23:31:17.502+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.502+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.502+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.502+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0c000 con 0x3e4a000 2026-02-11T23:31:17.502+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3d88a4f9:::obj551:head)]) -- 0x4e5dc80 con 0x3e4a000 2026-02-11T23:31:17.502+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.503+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 350 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.503+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.503+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.503+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 351 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3d88a4f9:::obj551:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3e4a000 2026-02-11T23:31:17.503+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3d88a4f9:::obj551:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.503+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3d88a4f9:::obj551:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3d88a4f9:::obj551:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3e07a120:::obj540:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3e07a120:::obj540:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3e07a120:::obj540:head 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3e07a120:::obj540:head v 107'540 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.503+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3e07a120:::obj540:head, version: 107'540, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3e07a120:::obj540:head@107'540, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3d88a4f9:::obj551:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.503+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3d88a4f9:::obj551:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 114/114 lb 4:3d88a4f9:::obj551:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.503+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.503+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3d88a4f9:::obj551:head) v3 2026-02-11T23:31:17.503+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d8750d8:::obj471:head local-lis/les=112/113 n=151 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3d88a4f9:::obj551:head) 2026-02-11T23:31:17.503+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.503+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3d88a4f9:::obj551:head) v3 finish latency 0.000017 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.503+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.503+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0c180 con 0x3e4a000 2026-02-11T23:31:17.503+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3e07a120:::obj540:head)]) -- 0x4c80600 con 0x3e4a000 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.503+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 352 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3e4a000 2026-02-11T23:31:17.503+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.503+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.503+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 353 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3e07a120:::obj540:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3e4a000 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.503+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3e07a120:::obj540:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 114 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.503+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3e07a120:::obj540:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 114/112 [PushOp(4:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3e07a120:::obj540:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3ed3e33d:::obj376:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3ed3e33d:::obj376:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3ed3e33d:::obj376:head 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3ed3e33d:::obj376:head v 107'376 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.503+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.503+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3ed3e33d:::obj376:head, version: 107'376, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3ed3e33d:::obj376:head@107'376, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.503+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3e07a120:::obj540:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.503+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3e07a120:::obj540:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 114/114 lb 4:3e07a120:::obj540:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.503+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.503+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3e07a120:::obj540:head) v3 2026-02-11T23:31:17.503+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3d88a4f9:::obj551:head local-lis/les=112/113 n=152 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3e07a120:::obj540:head) 2026-02-11T23:31:17.503+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.503+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3e07a120:::obj540:head) v3 finish latency 0.000014 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.503+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.504+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.504+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.504+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0c300 con 0x3e4a000 2026-02-11T23:31:17.504+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3ed3e33d:::obj376:head)]) -- 0x4b7c180 con 0x3e4a000 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.504+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 354 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.504+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.504+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.504+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.504+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 355 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3ed3e33d:::obj376:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3e4a000 2026-02-11T23:31:17.504+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3ed3e33d:::obj376:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.504+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.504+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3ed3e33d:::obj376:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.504+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.504+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.504+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 114/112 [PushOp(4:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.504+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3ed3e33d:::obj376:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.504+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.504+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.504+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3efbd745:::obj84:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.504+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3efbd745:::obj84:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.504+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3efbd745:::obj84:head 2026-02-11T23:31:17.504+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3efbd745:::obj84:head v 107'84 2026-02-11T23:31:17.504+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.504+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.504+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3efbd745:::obj84:head, version: 107'84, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3efbd745:::obj84:head@107'84, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.504+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3ed3e33d:::obj376:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.504+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3ed3e33d:::obj376:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 114/114 lb 4:3ed3e33d:::obj376:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.504+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.504+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3ed3e33d:::obj376:head) v3 2026-02-11T23:31:17.504+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3e07a120:::obj540:head local-lis/les=112/113 n=153 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3ed3e33d:::obj376:head) 2026-02-11T23:31:17.504+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.504+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3ed3e33d:::obj376:head) v3 finish latency 0.000013 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.504+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.504+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.504+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0c480 con 0x3e4a000 2026-02-11T23:31:17.504+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3efbd745:::obj84:head)]) -- 0x4b7d380 con 0x3e4a000 2026-02-11T23:31:17.504+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.505+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 356 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3e4a000 2026-02-11T23:31:17.505+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.505+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.505+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 357 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3efbd745:::obj84:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5080 con 0x3e4a000 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.505+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3efbd745:::obj84:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.505+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3efbd745:::obj84:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 114/112 [PushOp(4:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3efbd745:::obj84:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3fc284d5:::obj140:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3fc284d5:::obj140:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3fc284d5:::obj140:head 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3fc284d5:::obj140:head v 107'140 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.505+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3fc284d5:::obj140:head, version: 107'140, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fc284d5:::obj140:head@107'140, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3efbd745:::obj84:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3efbd745:::obj84:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 114/114 lb 4:3efbd745:::obj84:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3efbd745:::obj84:head) v3 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3ed3e33d:::obj376:head local-lis/les=112/113 n=154 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3efbd745:::obj84:head) 2026-02-11T23:31:17.505+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3efbd745:::obj84:head) v3 finish latency 0.000010 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.505+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0c600 con 0x3e4a000 2026-02-11T23:31:17.505+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3fc284d5:::obj140:head)]) -- 0x4c97980 con 0x3e4a000 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.505+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 358 ==== MOSDPGPush(4.0 114/112 [PushOp(4:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3e4a000 2026-02-11T23:31:17.505+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.505+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.505+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 359 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3fc284d5:::obj140:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3e4a000 2026-02-11T23:31:17.505+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3fc284d5:::obj140:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 114 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.505+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3fc284d5:::obj140:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 114/112 [PushOp(4:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3fc284d5:::obj140:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:3fe0a5f5:::obj46:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:3fe0a5f5:::obj46:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:3fe0a5f5:::obj46:head 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:3fe0a5f5:::obj46:head v 107'46 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.505+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.505+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:3fe0a5f5:::obj46:head, version: 107'46, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:3fe0a5f5:::obj46:head@107'46, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.505+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3fc284d5:::obj140:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3fc284d5:::obj140:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 114/114 lb 4:3fc284d5:::obj140:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3fc284d5:::obj140:head) v3 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3efbd745:::obj84:head local-lis/les=112/113 n=155 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3fc284d5:::obj140:head) 2026-02-11T23:31:17.505+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.505+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3fc284d5:::obj140:head) v3 finish latency 0.000019 2026-02-11T23:31:17.505+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.506+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.506+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.506+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0c900 con 0x3e4a000 2026-02-11T23:31:17.506+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:3fe0a5f5:::obj46:head)]) -- 0x4e0c780 con 0x3e4a000 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.506+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 360 ==== MOSDPGPush(4.0 114/112 [PushOp(4:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.506+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.506+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.506+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 361 ==== pg_backfill(progress 4.0 e 114/114 lb 4:3fe0a5f5:::obj46:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.506+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:3fe0a5f5:::obj46:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.506+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3fe0a5f5:::obj46:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.506+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 114/112 [PushOp(4:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.506+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.506+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.506+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.506+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3fe0a5f5:::obj46:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.506+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.506+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:401e0147:::obj213:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:401e0147:::obj213:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.506+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:401e0147:::obj213:head 2026-02-11T23:31:17.506+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:401e0147:::obj213:head v 107'213 2026-02-11T23:31:17.506+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.506+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.506+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:401e0147:::obj213:head, version: 107'213, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e0147:::obj213:head@107'213, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.506+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.506+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:3fe0a5f5:::obj46:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.506+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3fe0a5f5:::obj46:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 114/114 lb 4:3fe0a5f5:::obj46:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.506+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.506+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:3fe0a5f5:::obj46:head) v3 2026-02-11T23:31:17.506+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fc284d5:::obj140:head local-lis/les=112/113 n=156 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:3fe0a5f5:::obj46:head) 2026-02-11T23:31:17.506+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.506+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:3fe0a5f5:::obj46:head) v3 finish latency 0.000025 2026-02-11T23:31:17.506+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.506+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.506+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.506+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0cd80 con 0x3e4a000 2026-02-11T23:31:17.506+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:401e0147:::obj213:head)]) -- 0x4e0ca80 con 0x3e4a000 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.506+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.506+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.507+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 362 ==== MOSDPGPush(4.0 114/112 [PushOp(4:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.507+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.507+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.507+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.507+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 363 ==== pg_backfill(progress 4.0 e 114/114 lb 4:401e0147:::obj213:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5600 con 0x3e4a000 2026-02-11T23:31:17.507+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:401e0147:::obj213:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.507+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:401e0147:::obj213:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.507+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.507+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.507+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.507+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 114/112 [PushOp(4:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.507+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.507+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:401e0147:::obj213:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.507+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.507+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:401e941b:::obj398:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.507+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:401e941b:::obj398:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.507+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:401e941b:::obj398:head 2026-02-11T23:31:17.507+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:401e941b:::obj398:head v 107'398 2026-02-11T23:31:17.507+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.507+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.507+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:401e941b:::obj398:head, version: 107'398, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:401e941b:::obj398:head@107'398, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.507+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:401e0147:::obj213:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.507+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:401e0147:::obj213:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 114/114 lb 4:401e0147:::obj213:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.507+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.507+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:401e0147:::obj213:head) v3 2026-02-11T23:31:17.507+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:3fe0a5f5:::obj46:head local-lis/les=112/113 n=157 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:401e0147:::obj213:head) 2026-02-11T23:31:17.507+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.507+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:401e0147:::obj213:head) v3 finish latency 0.000014 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.507+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.507+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.507+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0cf00 con 0x3e4a000 2026-02-11T23:31:17.507+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:401e941b:::obj398:head)]) -- 0x4b7c300 con 0x3e4a000 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.507+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 364 ==== MOSDPGPush(4.0 114/112 [PushOp(4:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.507+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.507+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.507+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.507+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 365 ==== pg_backfill(progress 4.0 e 114/114 lb 4:401e941b:::obj398:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.507+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:401e941b:::obj398:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.507+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.507+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:401e941b:::obj398:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.507+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:401e941b:::obj398:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 114/112 [PushOp(4:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:40d1abdf:::obj254:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:40d1abdf:::obj254:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:40d1abdf:::obj254:head 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:40d1abdf:::obj254:head v 107'254 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.508+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:40d1abdf:::obj254:head, version: 107'254, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:40d1abdf:::obj254:head@107'254, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:401e941b:::obj398:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:401e941b:::obj398:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 114/114 lb 4:401e941b:::obj398:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:401e941b:::obj398:head) v3 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e0147:::obj213:head local-lis/les=112/113 n=158 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:401e941b:::obj398:head) 2026-02-11T23:31:17.508+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:401e941b:::obj398:head) v3 finish latency 0.000011 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.508+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0d080 con 0x3e4a000 2026-02-11T23:31:17.508+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:40d1abdf:::obj254:head)]) -- 0x4c97e00 con 0x3e4a000 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.508+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 366 ==== MOSDPGPush(4.0 114/112 [PushOp(4:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.508+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.508+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.508+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 367 ==== pg_backfill(progress 4.0 e 114/114 lb 4:40d1abdf:::obj254:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.508+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:40d1abdf:::obj254:head) v3 prio 15 type 95 cost 0 latency 0.000013 epoch 114 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.508+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:40d1abdf:::obj254:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 114/112 [PushOp(4:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:40d1abdf:::obj254:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:41892471:::obj524:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:41892471:::obj524:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:41892471:::obj524:head 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:41892471:::obj524:head v 107'524 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.508+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.508+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:41892471:::obj524:head, version: 107'524, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41892471:::obj524:head@107'524, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:40d1abdf:::obj254:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:40d1abdf:::obj254:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 114/114 lb 4:40d1abdf:::obj254:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:40d1abdf:::obj254:head) v3 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:401e941b:::obj398:head local-lis/les=112/113 n=159 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:40d1abdf:::obj254:head) 2026-02-11T23:31:17.508+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:40d1abdf:::obj254:head) v3 finish latency 0.000011 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.508+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.508+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.508+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0d200 con 0x3e4a000 2026-02-11T23:31:17.508+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:41892471:::obj524:head)]) -- 0x4b7c480 con 0x3e4a000 2026-02-11T23:31:17.508+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.509+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 368 ==== MOSDPGPush(4.0 114/112 [PushOp(4:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3e4a000 2026-02-11T23:31:17.509+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.509+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.509+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 369 ==== pg_backfill(progress 4.0 e 114/114 lb 4:41892471:::obj524:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.509+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:41892471:::obj524:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 114 2026-02-11T23:31:17.509+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.509+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:41892471:::obj524:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.509+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.509+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.509+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.509+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 114/112 [PushOp(4:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.509+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.509+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.509+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:41892471:::obj524:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.509+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.509+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.509+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:41ea52ba:::obj210:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.509+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:41ea52ba:::obj210:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.509+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:41ea52ba:::obj210:head 2026-02-11T23:31:17.509+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:41ea52ba:::obj210:head v 107'210 2026-02-11T23:31:17.509+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.509+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.509+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:41ea52ba:::obj210:head, version: 107'210, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:41ea52ba:::obj210:head@107'210, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.509+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.509+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.509+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:41892471:::obj524:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.509+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:41892471:::obj524:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 114/114 lb 4:41892471:::obj524:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.509+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.509+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:41892471:::obj524:head) v3 2026-02-11T23:31:17.509+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:40d1abdf:::obj254:head local-lis/les=112/113 n=160 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:41892471:::obj524:head) 2026-02-11T23:31:17.509+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.509+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:41892471:::obj524:head) v3 finish latency 0.000011 2026-02-11T23:31:17.509+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.509+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.509+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.509+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.509+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.509+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.509+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.509+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0d380 con 0x3e4a000 2026-02-11T23:31:17.509+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:41ea52ba:::obj210:head)]) -- 0x4fcfe00 con 0x3e4a000 2026-02-11T23:31:17.509+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.509+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 370 ==== MOSDPGPush(4.0 114/112 [PushOp(4:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea1a0 con 0x3e4a000 2026-02-11T23:31:17.509+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.509+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.509+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 371 ==== pg_backfill(progress 4.0 e 114/114 lb 4:41ea52ba:::obj210:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.509+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.509+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:41ea52ba:::obj210:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.509+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:41ea52ba:::obj210:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.509+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.509+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.509+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 114/112 [PushOp(4:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:41ea52ba:::obj210:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:424ce90f:::obj439:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.510+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:424ce90f:::obj439:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:424ce90f:::obj439:head 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:424ce90f:::obj439:head v 107'439 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.510+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:424ce90f:::obj439:head, version: 107'439, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:424ce90f:::obj439:head@107'439, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.510+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:41ea52ba:::obj210:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:41ea52ba:::obj210:head) v3 prio 15 cost 0 latency 0.000100 pg_backfill(progress 4.0 e 114/114 lb 4:41ea52ba:::obj210:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:41ea52ba:::obj210:head) v3 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41892471:::obj524:head local-lis/les=112/113 n=161 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:41ea52ba:::obj210:head) 2026-02-11T23:31:17.510+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:41ea52ba:::obj210:head) v3 finish latency 0.000010 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.510+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.510+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0d500 con 0x3e4a000 2026-02-11T23:31:17.510+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:424ce90f:::obj439:head)]) -- 0x4b7c780 con 0x3e4a000 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.510+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 372 ==== MOSDPGPush(4.0 114/112 [PushOp(4:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3e4a000 2026-02-11T23:31:17.510+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 114 2026-02-11T23:31:17.510+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.510+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.510+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 373 ==== pg_backfill(progress 4.0 e 114/114 lb 4:424ce90f:::obj439:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.510+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:424ce90f:::obj439:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.510+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.510+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:424ce90f:::obj439:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.510+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.510+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 114/112 [PushOp(4:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:424ce90f:::obj439:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:42d0e5ef:::obj468:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.510+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:42d0e5ef:::obj468:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:42d0e5ef:::obj468:head 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:42d0e5ef:::obj468:head v 107'468 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.510+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.510+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:42d0e5ef:::obj468:head, version: 107'468, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:42d0e5ef:::obj468:head@107'468, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.510+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:424ce90f:::obj439:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:424ce90f:::obj439:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 114/114 lb 4:424ce90f:::obj439:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:424ce90f:::obj439:head) v3 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:41ea52ba:::obj210:head local-lis/les=112/113 n=162 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:424ce90f:::obj439:head) 2026-02-11T23:31:17.510+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:424ce90f:::obj439:head) v3 finish latency 0.000009 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.510+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.510+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.510+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0d680 con 0x3e4a000 2026-02-11T23:31:17.510+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:42d0e5ef:::obj468:head)]) -- 0x4c96000 con 0x3e4a000 2026-02-11T23:31:17.510+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.511+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 374 ==== MOSDPGPush(4.0 114/112 [PushOp(4:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dd40 con 0x3e4a000 2026-02-11T23:31:17.511+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.511+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.511+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 375 ==== pg_backfill(progress 4.0 e 114/114 lb 4:42d0e5ef:::obj468:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.511+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.511+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:42d0e5ef:::obj468:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.511+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:42d0e5ef:::obj468:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.511+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.511+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.511+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.511+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 114/112 [PushOp(4:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.511+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:42d0e5ef:::obj468:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.511+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.511+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.511+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:436594ec:::obj300:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.511+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:436594ec:::obj300:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.511+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:436594ec:::obj300:head 2026-02-11T23:31:17.511+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:436594ec:::obj300:head v 107'300 2026-02-11T23:31:17.511+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.511+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.511+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:436594ec:::obj300:head, version: 107'300, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:436594ec:::obj300:head@107'300, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.511+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:42d0e5ef:::obj468:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:42d0e5ef:::obj468:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 114/114 lb 4:42d0e5ef:::obj468:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:42d0e5ef:::obj468:head) v3 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:424ce90f:::obj439:head local-lis/les=112/113 n=163 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:42d0e5ef:::obj468:head) 2026-02-11T23:31:17.511+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:42d0e5ef:::obj468:head) v3 finish latency 0.000011 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.511+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.511+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0d800 con 0x3e4a000 2026-02-11T23:31:17.511+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:436594ec:::obj300:head)]) -- 0x4b7c900 con 0x3e4a000 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.511+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 376 ==== MOSDPGPush(4.0 114/112 [PushOp(4:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183380 con 0x3e4a000 2026-02-11T23:31:17.511+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.511+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.511+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 377 ==== pg_backfill(progress 4.0 e 114/114 lb 4:436594ec:::obj300:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.511+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:436594ec:::obj300:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 114 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.511+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:436594ec:::obj300:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 114/112 [PushOp(4:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.511+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.511+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:436594ec:::obj300:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:454d8265:::obj256:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.511+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:454d8265:::obj256:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:454d8265:::obj256:head 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:454d8265:::obj256:head v 107'256 2026-02-11T23:31:17.511+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.511+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.512+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:454d8265:::obj256:head, version: 107'256, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:454d8265:::obj256:head@107'256, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.512+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.512+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:436594ec:::obj300:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:436594ec:::obj300:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 114/114 lb 4:436594ec:::obj300:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:436594ec:::obj300:head) v3 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:42d0e5ef:::obj468:head local-lis/les=112/113 n=164 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:436594ec:::obj300:head) 2026-02-11T23:31:17.512+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:436594ec:::obj300:head) v3 finish latency 0.000019 2026-02-11T23:31:17.512+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.512+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.512+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.512+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0dc80 con 0x3e4a000 2026-02-11T23:31:17.512+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:454d8265:::obj256:head)]) -- 0x4e0d980 con 0x3e4a000 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.512+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 378 ==== MOSDPGPush(4.0 114/112 [PushOp(4:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4de44e0 con 0x3e4a000 2026-02-11T23:31:17.512+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.512+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.512+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 379 ==== pg_backfill(progress 4.0 e 114/114 lb 4:454d8265:::obj256:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3e4a000 2026-02-11T23:31:17.512+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:454d8265:::obj256:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.512+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.512+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:454d8265:::obj256:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.512+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.512+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.512+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 114/112 [PushOp(4:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:454d8265:::obj256:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:461f8b5e:::obj16:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.512+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:461f8b5e:::obj16:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:461f8b5e:::obj16:head 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:461f8b5e:::obj16:head v 107'16 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.512+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.512+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:461f8b5e:::obj16:head, version: 107'16, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:461f8b5e:::obj16:head@107'16, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.512+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:454d8265:::obj256:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.512+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:454d8265:::obj256:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 114/114 lb 4:454d8265:::obj256:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.512+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.512+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:454d8265:::obj256:head) v3 2026-02-11T23:31:17.512+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:436594ec:::obj300:head local-lis/les=112/113 n=165 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:454d8265:::obj256:head) 2026-02-11T23:31:17.512+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.512+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:454d8265:::obj256:head) v3 finish latency 0.000015 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.512+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.512+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.512+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0de00 con 0x3e4a000 2026-02-11T23:31:17.512+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:461f8b5e:::obj16:head)]) -- 0x4c97800 con 0x3e4a000 2026-02-11T23:31:17.512+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.513+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 380 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d040 con 0x3e4a000 2026-02-11T23:31:17.513+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.513+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.513+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 381 ==== pg_backfill(progress 4.0 e 114/114 lb 4:461f8b5e:::obj16:head) ==== 1061+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.513+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:461f8b5e:::obj16:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.513+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:461f8b5e:::obj16:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 114/112 [PushOp(4:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:461f8b5e:::obj16:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4667f6bf:::obj584:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4667f6bf:::obj584:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4667f6bf:::obj584:head 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4667f6bf:::obj584:head v 107'584 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.513+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4667f6bf:::obj584:head, version: 107'584, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4667f6bf:::obj584:head@107'584, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:461f8b5e:::obj16:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:461f8b5e:::obj16:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 114/114 lb 4:461f8b5e:::obj16:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:461f8b5e:::obj16:head) v3 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:454d8265:::obj256:head local-lis/les=112/113 n=166 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:461f8b5e:::obj16:head) 2026-02-11T23:31:17.513+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:461f8b5e:::obj16:head) v3 finish latency 0.000024 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.513+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0c480 con 0x3e4a000 2026-02-11T23:31:17.513+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4667f6bf:::obj584:head)]) -- 0x4e0c600 con 0x3e4a000 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.513+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 382 ==== MOSDPGPush(4.0 114/112 [PushOp(4:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.513+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.513+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.513+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 383 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4667f6bf:::obj584:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3e4a000 2026-02-11T23:31:17.513+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4667f6bf:::obj584:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 114 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.513+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4667f6bf:::obj584:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 114/112 [PushOp(4:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4667f6bf:::obj584:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:46b5a6d0:::obj580:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:46b5a6d0:::obj580:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:46b5a6d0:::obj580:head 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:46b5a6d0:::obj580:head v 107'580 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.513+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.513+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:46b5a6d0:::obj580:head, version: 107'580, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:46b5a6d0:::obj580:head@107'580, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.513+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.513+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4667f6bf:::obj584:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4667f6bf:::obj584:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 114/114 lb 4:4667f6bf:::obj584:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4667f6bf:::obj584:head) v3 2026-02-11T23:31:17.513+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:461f8b5e:::obj16:head local-lis/les=112/113 n=167 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4667f6bf:::obj584:head) 2026-02-11T23:31:17.514+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4667f6bf:::obj584:head) v3 finish latency 0.000019 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.514+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.514+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4b7c180 con 0x3e4a000 2026-02-11T23:31:17.514+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:46b5a6d0:::obj580:head)]) -- 0x4c81680 con 0x3e4a000 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.514+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 384 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.514+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.514+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.514+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 385 ==== pg_backfill(progress 4.0 e 114/114 lb 4:46b5a6d0:::obj580:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bc100 con 0x3e4a000 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.514+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:46b5a6d0:::obj580:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 114 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.514+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:46b5a6d0:::obj580:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.514+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 114/112 [PushOp(4:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.514+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.514+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:46b5a6d0:::obj580:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4851a45a:::obj44:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4851a45a:::obj44:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4851a45a:::obj44:head 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4851a45a:::obj44:head v 107'44 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.514+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4851a45a:::obj44:head, version: 107'44, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4851a45a:::obj44:head@107'44, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.514+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.514+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:46b5a6d0:::obj580:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.514+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:46b5a6d0:::obj580:head) v3 prio 15 cost 0 latency 0.000088 pg_backfill(progress 4.0 e 114/114 lb 4:46b5a6d0:::obj580:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.514+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.514+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:46b5a6d0:::obj580:head) v3 2026-02-11T23:31:17.514+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4667f6bf:::obj584:head local-lis/les=112/113 n=168 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:46b5a6d0:::obj580:head) 2026-02-11T23:31:17.514+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.514+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:46b5a6d0:::obj580:head) v3 finish latency 0.000017 2026-02-11T23:31:17.514+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.514+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.514+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.514+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c80600 con 0x3e4a000 2026-02-11T23:31:17.514+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4851a45a:::obj44:head)]) -- 0x4e0c300 con 0x3e4a000 2026-02-11T23:31:17.514+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.515+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 386 ==== MOSDPGPush(4.0 114/112 [PushOp(4:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.515+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.515+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.515+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 387 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4851a45a:::obj44:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3e4a000 2026-02-11T23:31:17.515+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4851a45a:::obj44:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 114 2026-02-11T23:31:17.515+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4851a45a:::obj44:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 114/112 [PushOp(4:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4851a45a:::obj44:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:48c1458b:::obj274:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:48c1458b:::obj274:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:48c1458b:::obj274:head 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:48c1458b:::obj274:head v 107'274 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.515+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:48c1458b:::obj274:head, version: 107'274, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:48c1458b:::obj274:head@107'274, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4851a45a:::obj44:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.515+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4851a45a:::obj44:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 114/114 lb 4:4851a45a:::obj44:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.515+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.515+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4851a45a:::obj44:head) v3 2026-02-11T23:31:17.515+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:46b5a6d0:::obj580:head local-lis/les=112/113 n=169 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4851a45a:::obj44:head) 2026-02-11T23:31:17.515+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.515+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4851a45a:::obj44:head) v3 finish latency 0.000013 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.515+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.515+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0c180 con 0x3e4a000 2026-02-11T23:31:17.515+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:48c1458b:::obj274:head)]) -- 0x4c97680 con 0x3e4a000 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.515+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 388 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.515+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.515+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.515+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 389 ==== pg_backfill(progress 4.0 e 114/114 lb 4:48c1458b:::obj274:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3e4a000 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.515+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:48c1458b:::obj274:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 114 2026-02-11T23:31:17.515+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:48c1458b:::obj274:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000040 MOSDPGPush(4.0 114/112 [PushOp(4:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:48c1458b:::obj274:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4911cfa9:::obj108:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4911cfa9:::obj108:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4911cfa9:::obj108:head 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4911cfa9:::obj108:head v 107'108 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.515+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.515+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4911cfa9:::obj108:head, version: 107'108, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4911cfa9:::obj108:head@107'108, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.515+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:48c1458b:::obj274:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.515+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:48c1458b:::obj274:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 114/114 lb 4:48c1458b:::obj274:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.515+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.515+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:48c1458b:::obj274:head) v3 2026-02-11T23:31:17.515+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4851a45a:::obj44:head local-lis/les=112/113 n=170 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:48c1458b:::obj274:head) 2026-02-11T23:31:17.515+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.515+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:48c1458b:::obj274:head) v3 finish latency 0.000012 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.515+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.516+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.516+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.516+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5dc80 con 0x3e4a000 2026-02-11T23:31:17.516+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4911cfa9:::obj108:head)]) -- 0x4d8e180 con 0x3e4a000 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.516+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 390 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ebba0 con 0x3e4a000 2026-02-11T23:31:17.516+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.516+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.516+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.516+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 391 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4911cfa9:::obj108:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3e4a000 2026-02-11T23:31:17.516+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4911cfa9:::obj108:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 114 2026-02-11T23:31:17.516+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.516+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4911cfa9:::obj108:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.516+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.516+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.516+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 114/112 [PushOp(4:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.516+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4911cfa9:::obj108:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.516+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.516+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.516+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4918aa91:::obj303:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.516+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4918aa91:::obj303:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.516+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4918aa91:::obj303:head 2026-02-11T23:31:17.516+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4918aa91:::obj303:head v 107'303 2026-02-11T23:31:17.516+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.516+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.516+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4918aa91:::obj303:head, version: 107'303, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4918aa91:::obj303:head@107'303, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.516+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4911cfa9:::obj108:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.516+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4911cfa9:::obj108:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 114/114 lb 4:4911cfa9:::obj108:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.516+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.516+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4911cfa9:::obj108:head) v3 2026-02-11T23:31:17.516+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:48c1458b:::obj274:head local-lis/les=112/113 n=171 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4911cfa9:::obj108:head) 2026-02-11T23:31:17.516+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.516+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4911cfa9:::obj108:head) v3 finish latency 0.000009 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.516+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.516+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.516+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0c000 con 0x3e4a000 2026-02-11T23:31:17.516+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4918aa91:::obj303:head)]) -- 0x4d8e480 con 0x3e4a000 2026-02-11T23:31:17.516+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.517+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 392 ==== MOSDPGPush(4.0 114/112 [PushOp(4:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3e4a000 2026-02-11T23:31:17.517+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.517+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.517+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 393 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4918aa91:::obj303:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3e4a000 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.517+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4918aa91:::obj303:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 114 2026-02-11T23:31:17.517+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4918aa91:::obj303:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 114/112 [PushOp(4:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4918aa91:::obj303:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:492a75ed:::obj72:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:492a75ed:::obj72:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:492a75ed:::obj72:head 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:492a75ed:::obj72:head v 107'72 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.517+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:492a75ed:::obj72:head, version: 107'72, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:492a75ed:::obj72:head@107'72, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4918aa91:::obj303:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4918aa91:::obj303:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 114/114 lb 4:4918aa91:::obj303:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4918aa91:::obj303:head) v3 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4911cfa9:::obj108:head local-lis/les=112/113 n=172 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4918aa91:::obj303:head) 2026-02-11T23:31:17.517+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4918aa91:::obj303:head) v3 finish latency 0.000009 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.517+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4b7d500 con 0x3e4a000 2026-02-11T23:31:17.517+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:492a75ed:::obj72:head)]) -- 0x4d8e780 con 0x3e4a000 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.517+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 394 ==== MOSDPGPush(4.0 114/112 [PushOp(4:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3e4a000 2026-02-11T23:31:17.517+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.517+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.517+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 395 ==== pg_backfill(progress 4.0 e 114/114 lb 4:492a75ed:::obj72:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3e4a000 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.517+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:492a75ed:::obj72:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 114 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.517+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:492a75ed:::obj72:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000038 MOSDPGPush(4.0 114/112 [PushOp(4:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:492a75ed:::obj72:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:498092df:::obj260:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:498092df:::obj260:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:498092df:::obj260:head 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:498092df:::obj260:head v 107'260 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.517+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.517+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:498092df:::obj260:head, version: 107'260, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498092df:::obj260:head@107'260, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:492a75ed:::obj72:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:492a75ed:::obj72:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 114/114 lb 4:492a75ed:::obj72:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:492a75ed:::obj72:head) v3 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4918aa91:::obj303:head local-lis/les=112/113 n=173 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:492a75ed:::obj72:head) 2026-02-11T23:31:17.517+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:492a75ed:::obj72:head) v3 finish latency 0.000009 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.517+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.517+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.517+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5db00 con 0x3e4a000 2026-02-11T23:31:17.517+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:498092df:::obj260:head)]) -- 0x4d8e900 con 0x3e4a000 2026-02-11T23:31:17.517+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.518+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 396 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3e4a000 2026-02-11T23:31:17.518+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.518+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.518+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 397 ==== pg_backfill(progress 4.0 e 114/114 lb 4:498092df:::obj260:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3e4a000 2026-02-11T23:31:17.518+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.518+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:498092df:::obj260:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.518+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:498092df:::obj260:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.518+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.518+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.518+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.518+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 114/112 [PushOp(4:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.518+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.518+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.518+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.518+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.518+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:498092df:::obj260:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.518+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4987b93a:::obj56:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.518+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4987b93a:::obj56:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.518+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4987b93a:::obj56:head 2026-02-11T23:31:17.518+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4987b93a:::obj56:head v 107'56 2026-02-11T23:31:17.518+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.518+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.518+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4987b93a:::obj56:head, version: 107'56, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4987b93a:::obj56:head@107'56, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.518+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.518+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.518+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.518+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:498092df:::obj260:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.518+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:498092df:::obj260:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 114/114 lb 4:498092df:::obj260:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.518+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.518+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:498092df:::obj260:head) v3 2026-02-11T23:31:17.518+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:492a75ed:::obj72:head local-lis/les=112/113 n=174 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:498092df:::obj260:head) 2026-02-11T23:31:17.518+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.518+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:498092df:::obj260:head) v3 finish latency 0.000019 2026-02-11T23:31:17.518+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.518+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.518+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.518+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.518+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.518+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.518+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d980 con 0x3e4a000 2026-02-11T23:31:17.518+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4987b93a:::obj56:head)]) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.518+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.519+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 398 ==== MOSDPGPush(4.0 114/112 [PushOp(4:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3e4a000 2026-02-11T23:31:17.519+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.519+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.519+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 399 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4987b93a:::obj56:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3e4a000 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.519+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4987b93a:::obj56:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.519+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4987b93a:::obj56:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 114/112 [PushOp(4:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:498e91f7:::obj463:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4987b93a:::obj56:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:498e91f7:::obj463:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:498e91f7:::obj463:head 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:498e91f7:::obj463:head v 107'463 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.519+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:498e91f7:::obj463:head, version: 107'463, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:498e91f7:::obj463:head@107'463, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4987b93a:::obj56:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4987b93a:::obj56:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 114/114 lb 4:4987b93a:::obj56:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4987b93a:::obj56:head) v3 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498092df:::obj260:head local-lis/les=112/113 n=175 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4987b93a:::obj56:head) 2026-02-11T23:31:17.519+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4987b93a:::obj56:head) v3 finish latency 0.000034 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.519+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d500 con 0x3e4a000 2026-02-11T23:31:17.519+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:498e91f7:::obj463:head)]) -- 0x4fcf200 con 0x3e4a000 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.519+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 400 ==== MOSDPGPush(4.0 114/112 [PushOp(4:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2340 con 0x3e4a000 2026-02-11T23:31:17.519+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.519+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.519+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 401 ==== pg_backfill(progress 4.0 e 114/114 lb 4:498e91f7:::obj463:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3e4a000 2026-02-11T23:31:17.519+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:498e91f7:::obj463:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.519+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:498e91f7:::obj463:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 114/112 [PushOp(4:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:498e91f7:::obj463:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:49f89eae:::obj182:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:49f89eae:::obj182:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:49f89eae:::obj182:head 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:49f89eae:::obj182:head v 107'182 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.519+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.519+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:49f89eae:::obj182:head, version: 107'182, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:49f89eae:::obj182:head@107'182, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:498e91f7:::obj463:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:498e91f7:::obj463:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 114/114 lb 4:498e91f7:::obj463:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:498e91f7:::obj463:head) v3 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4987b93a:::obj56:head local-lis/les=112/113 n=176 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:498e91f7:::obj463:head) 2026-02-11T23:31:17.519+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:498e91f7:::obj463:head) v3 finish latency 0.000015 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.519+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.519+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.519+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d680 con 0x3e4a000 2026-02-11T23:31:17.519+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:49f89eae:::obj182:head)]) -- 0x4d8ec00 con 0x3e4a000 2026-02-11T23:31:17.519+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.520+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 402 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.520+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.520+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.520+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 403 ==== pg_backfill(progress 4.0 e 114/114 lb 4:49f89eae:::obj182:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3e4a000 2026-02-11T23:31:17.520+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:49f89eae:::obj182:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 114 2026-02-11T23:31:17.520+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:49f89eae:::obj182:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.520+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 114/112 [PushOp(4:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.520+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.520+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.520+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.520+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:49f89eae:::obj182:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.520+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.520+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4b1d3252:::obj86:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4b1d3252:::obj86:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.520+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4b1d3252:::obj86:head 2026-02-11T23:31:17.520+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4b1d3252:::obj86:head v 107'86 2026-02-11T23:31:17.520+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.520+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.520+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4b1d3252:::obj86:head, version: 107'86, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b1d3252:::obj86:head@107'86, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.520+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.520+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:49f89eae:::obj182:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.520+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:49f89eae:::obj182:head) v3 prio 15 cost 0 latency 0.000091 pg_backfill(progress 4.0 e 114/114 lb 4:49f89eae:::obj182:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.520+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.520+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:49f89eae:::obj182:head) v3 2026-02-11T23:31:17.520+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:498e91f7:::obj463:head local-lis/les=112/113 n=177 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:49f89eae:::obj182:head) 2026-02-11T23:31:17.520+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.520+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:49f89eae:::obj182:head) v3 finish latency 0.000036 2026-02-11T23:31:17.520+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.520+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.520+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.520+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d380 con 0x3e4a000 2026-02-11T23:31:17.520+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4b1d3252:::obj86:head)]) -- 0x49d7800 con 0x3e4a000 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.520+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.520+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.520+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 404 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.520+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.520+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.521+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 405 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4b1d3252:::obj86:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3e4a000 2026-02-11T23:31:17.521+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4b1d3252:::obj86:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.521+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4b1d3252:::obj86:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 114/112 [PushOp(4:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4b1d3252:::obj86:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4b57ab44:::obj335:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4b57ab44:::obj335:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4b57ab44:::obj335:head 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4b57ab44:::obj335:head v 107'335 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.521+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4b57ab44:::obj335:head, version: 107'335, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b57ab44:::obj335:head@107'335, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000094 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4b1d3252:::obj86:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4b1d3252:::obj86:head) v3 prio 15 cost 0 latency 0.000131 pg_backfill(progress 4.0 e 114/114 lb 4:4b1d3252:::obj86:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4b1d3252:::obj86:head) v3 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:49f89eae:::obj182:head local-lis/les=112/113 n=178 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4b1d3252:::obj86:head) 2026-02-11T23:31:17.521+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4b1d3252:::obj86:head) v3 finish latency 0.000017 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.521+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.521+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4b57ab44:::obj335:head)]) -- 0x4d8ef00 con 0x3e4a000 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.521+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 406 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.521+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 114 2026-02-11T23:31:17.521+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.521+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 407 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4b57ab44:::obj335:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d700 con 0x3e4a000 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.521+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4b57ab44:::obj335:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 114 2026-02-11T23:31:17.521+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4b57ab44:::obj335:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 114/112 [PushOp(4:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4b57ab44:::obj335:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4b639beb:::obj280:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4b639beb:::obj280:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4b639beb:::obj280:head 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4b639beb:::obj280:head v 107'280 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.521+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.521+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4b639beb:::obj280:head, version: 107'280, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4b639beb:::obj280:head@107'280, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4b57ab44:::obj335:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4b57ab44:::obj335:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 114/114 lb 4:4b57ab44:::obj335:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4b57ab44:::obj335:head) v3 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b1d3252:::obj86:head local-lis/les=112/113 n=179 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4b57ab44:::obj335:head) 2026-02-11T23:31:17.521+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4b57ab44:::obj335:head) v3 finish latency 0.000014 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.521+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.521+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.521+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.521+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4b639beb:::obj280:head)]) -- 0x4d8f200 con 0x3e4a000 2026-02-11T23:31:17.521+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.522+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 408 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.522+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.522+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.522+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.522+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 409 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4b639beb:::obj280:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3e4a000 2026-02-11T23:31:17.522+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.522+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4b639beb:::obj280:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.522+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4b639beb:::obj280:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.522+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.522+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.522+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 114/112 [PushOp(4:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.522+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.522+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.522+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4b639beb:::obj280:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.522+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.522+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.522+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4baa8bb7:::obj594:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.522+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4baa8bb7:::obj594:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.522+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4baa8bb7:::obj594:head 2026-02-11T23:31:17.522+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4baa8bb7:::obj594:head v 107'594 2026-02-11T23:31:17.522+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.522+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.522+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4baa8bb7:::obj594:head, version: 107'594, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4baa8bb7:::obj594:head@107'594, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.522+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.522+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.522+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4b639beb:::obj280:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.522+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4b639beb:::obj280:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 114/114 lb 4:4b639beb:::obj280:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.522+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.522+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4b639beb:::obj280:head) v3 2026-02-11T23:31:17.522+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b57ab44:::obj335:head local-lis/les=112/113 n=180 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4b639beb:::obj280:head) 2026-02-11T23:31:17.522+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.522+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4b639beb:::obj280:head) v3 finish latency 0.000014 2026-02-11T23:31:17.522+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.522+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.522+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.522+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.522+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.522+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.522+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.522+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x49b8c00 con 0x3e4a000 2026-02-11T23:31:17.522+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4baa8bb7:::obj594:head)]) -- 0x4d8f500 con 0x3e4a000 2026-02-11T23:31:17.522+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.522+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 410 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d040 con 0x3e4a000 2026-02-11T23:31:17.522+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.522+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.522+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.522+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 411 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4baa8bb7:::obj594:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3e4a000 2026-02-11T23:31:17.522+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4baa8bb7:::obj594:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.522+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.522+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4baa8bb7:::obj594:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.522+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.522+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4baa8bb7:::obj594:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 114/112 [PushOp(4:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4c3f91c8:::obj470:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.523+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4c3f91c8:::obj470:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4c3f91c8:::obj470:head 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4c3f91c8:::obj470:head v 107'470 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.523+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c3f91c8:::obj470:head, version: 107'470, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c3f91c8:::obj470:head@107'470, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.523+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4baa8bb7:::obj594:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4baa8bb7:::obj594:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 114/114 lb 4:4baa8bb7:::obj594:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4baa8bb7:::obj594:head) v3 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4b639beb:::obj280:head local-lis/les=112/113 n=181 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4baa8bb7:::obj594:head) 2026-02-11T23:31:17.523+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4baa8bb7:::obj594:head) v3 finish latency 0.000016 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.523+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.523+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d080 con 0x3e4a000 2026-02-11T23:31:17.523+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4c3f91c8:::obj470:head)]) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.523+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 412 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de44e0 con 0x3e4a000 2026-02-11T23:31:17.523+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.523+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.523+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.523+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 413 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4c3f91c8:::obj470:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3e4a000 2026-02-11T23:31:17.523+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c3f91c8:::obj470:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 114 2026-02-11T23:31:17.523+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.523+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c3f91c8:::obj470:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.523+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.523+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 114/112 [PushOp(4:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c3f91c8:::obj470:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4c59dc70:::obj344:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.523+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4c59dc70:::obj344:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4c59dc70:::obj344:head 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4c59dc70:::obj344:head v 107'344 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.523+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.523+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c59dc70:::obj344:head, version: 107'344, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c59dc70:::obj344:head@107'344, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.523+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c3f91c8:::obj470:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c3f91c8:::obj470:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 114/114 lb 4:4c3f91c8:::obj470:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4c3f91c8:::obj470:head) v3 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4baa8bb7:::obj594:head local-lis/les=112/113 n=182 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4c3f91c8:::obj470:head) 2026-02-11T23:31:17.523+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c3f91c8:::obj470:head) v3 finish latency 0.000016 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.523+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.523+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.523+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4b7dc80 con 0x3e4a000 2026-02-11T23:31:17.523+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4c59dc70:::obj344:head)]) -- 0x4d8fb00 con 0x3e4a000 2026-02-11T23:31:17.523+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.524+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 414 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183380 con 0x3e4a000 2026-02-11T23:31:17.524+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.524+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.524+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 415 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4c59dc70:::obj344:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3e4a000 2026-02-11T23:31:17.524+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c59dc70:::obj344:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 114 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.524+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c59dc70:::obj344:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 114/112 [PushOp(4:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c59dc70:::obj344:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4c73ba95:::obj68:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4c73ba95:::obj68:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4c73ba95:::obj68:head 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4c73ba95:::obj68:head v 107'68 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.524+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c73ba95:::obj68:head, version: 107'68, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c73ba95:::obj68:head@107'68, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c59dc70:::obj344:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.524+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c59dc70:::obj344:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 114/114 lb 4:4c59dc70:::obj344:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.524+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.524+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4c59dc70:::obj344:head) v3 2026-02-11T23:31:17.524+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c3f91c8:::obj470:head local-lis/les=112/113 n=183 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4c59dc70:::obj344:head) 2026-02-11T23:31:17.524+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.524+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c59dc70:::obj344:head) v3 finish latency 0.000014 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.524+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.524+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5cf00 con 0x3e4a000 2026-02-11T23:31:17.524+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4c73ba95:::obj68:head)]) -- 0x4d8fe00 con 0x3e4a000 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.524+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 416 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dd40 con 0x3e4a000 2026-02-11T23:31:17.524+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.524+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.524+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 417 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4c73ba95:::obj68:head) ==== 1061+0+0 (unknown 0 0 0) 0x5162000 con 0x3e4a000 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.524+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c73ba95:::obj68:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.524+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c73ba95:::obj68:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 114/112 [PushOp(4:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.524+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c73ba95:::obj68:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4c7e2cfa:::obj454:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.524+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4c7e2cfa:::obj454:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4c7e2cfa:::obj454:head 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4c7e2cfa:::obj454:head v 107'454 2026-02-11T23:31:17.524+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.525+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.525+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c7e2cfa:::obj454:head, version: 107'454, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c7e2cfa:::obj454:head@107'454, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.525+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c73ba95:::obj68:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c73ba95:::obj68:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 114/114 lb 4:4c73ba95:::obj68:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4c73ba95:::obj68:head) v3 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c59dc70:::obj344:head local-lis/les=112/113 n=184 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4c73ba95:::obj68:head) 2026-02-11T23:31:17.525+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c73ba95:::obj68:head) v3 finish latency 0.000014 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.525+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.525+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c97200 con 0x3e4a000 2026-02-11T23:31:17.525+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4c7e2cfa:::obj454:head)]) -- 0x4e5dc80 con 0x3e4a000 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.525+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 418 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3e4a000 2026-02-11T23:31:17.525+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 114 2026-02-11T23:31:17.525+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.525+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.525+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 419 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4c7e2cfa:::obj454:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162580 con 0x3e4a000 2026-02-11T23:31:17.525+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c7e2cfa:::obj454:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.525+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.525+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c7e2cfa:::obj454:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.525+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.525+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.525+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 114/112 [PushOp(4:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c7e2cfa:::obj454:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.525+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.525+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.525+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.525+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4c9f7fa8:::obj477:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.525+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4c9f7fa8:::obj477:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.525+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4c9f7fa8:::obj477:head 2026-02-11T23:31:17.525+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4c9f7fa8:::obj477:head v 107'477 2026-02-11T23:31:17.525+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.525+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.525+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4c9f7fa8:::obj477:head, version: 107'477, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4c9f7fa8:::obj477:head@107'477, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.525+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c7e2cfa:::obj454:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c7e2cfa:::obj454:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 114/114 lb 4:4c7e2cfa:::obj454:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4c7e2cfa:::obj454:head) v3 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c73ba95:::obj68:head local-lis/les=112/113 n=185 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4c7e2cfa:::obj454:head) 2026-02-11T23:31:17.525+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c7e2cfa:::obj454:head) v3 finish latency 0.000015 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.525+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.525+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5cd80 con 0x3e4a000 2026-02-11T23:31:17.525+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.525+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4c9f7fa8:::obj477:head)]) -- 0x4e0c180 con 0x3e4a000 2026-02-11T23:31:17.525+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.526+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 420 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea1a0 con 0x3e4a000 2026-02-11T23:31:17.526+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.526+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.526+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 421 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4c9f7fa8:::obj477:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3e4a000 2026-02-11T23:31:17.526+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c9f7fa8:::obj477:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 114 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.526+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c9f7fa8:::obj477:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000039 MOSDPGPush(4.0 114/112 [PushOp(4:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c9f7fa8:::obj477:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4dc3027d:::obj550:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4dc3027d:::obj550:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4dc3027d:::obj550:head 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4dc3027d:::obj550:head v 107'550 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.526+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4dc3027d:::obj550:head, version: 107'550, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dc3027d:::obj550:head@107'550, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4c9f7fa8:::obj477:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.526+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c9f7fa8:::obj477:head) v3 prio 15 cost 0 latency 0.000092 pg_backfill(progress 4.0 e 114/114 lb 4:4c9f7fa8:::obj477:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.526+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.526+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4c9f7fa8:::obj477:head) v3 2026-02-11T23:31:17.526+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c7e2cfa:::obj454:head local-lis/les=112/113 n=186 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4c9f7fa8:::obj477:head) 2026-02-11T23:31:17.526+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.526+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4c9f7fa8:::obj477:head) v3 finish latency 0.000014 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.526+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.526+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4b7d800 con 0x3e4a000 2026-02-11T23:31:17.526+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4dc3027d:::obj550:head)]) -- 0x4c80600 con 0x3e4a000 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.526+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 422 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3e4a000 2026-02-11T23:31:17.526+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 114 2026-02-11T23:31:17.526+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.526+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 423 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4dc3027d:::obj550:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163080 con 0x3e4a000 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.526+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4dc3027d:::obj550:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.526+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4dc3027d:::obj550:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000037 MOSDPGPush(4.0 114/112 [PushOp(4:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4dc3027d:::obj550:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4dd77094:::obj38:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4dd77094:::obj38:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4dd77094:::obj38:head 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4dd77094:::obj38:head v 107'38 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.526+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.526+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4dd77094:::obj38:head, version: 107'38, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4dd77094:::obj38:head@107'38, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.526+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4dc3027d:::obj550:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.526+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4dc3027d:::obj550:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 114/114 lb 4:4dc3027d:::obj550:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.526+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.526+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4dc3027d:::obj550:head) v3 2026-02-11T23:31:17.526+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4c9f7fa8:::obj477:head local-lis/les=112/113 n=187 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4dc3027d:::obj550:head) 2026-02-11T23:31:17.526+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.526+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4dc3027d:::obj550:head) v3 finish latency 0.000014 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.526+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.527+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.527+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5cc00 con 0x3e4a000 2026-02-11T23:31:17.527+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.527+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4dd77094:::obj38:head)]) -- 0x4b7c180 con 0x3e4a000 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.527+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 424 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.527+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.527+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.527+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.527+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 425 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4dd77094:::obj38:head) ==== 1061+0+0 (unknown 0 0 0) 0x5163600 con 0x3e4a000 2026-02-11T23:31:17.527+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4dd77094:::obj38:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.527+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.527+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4dd77094:::obj38:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.527+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.527+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.527+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 114/112 [PushOp(4:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4dd77094:::obj38:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.527+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.527+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.527+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.527+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4eb30133:::obj30:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.527+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4eb30133:::obj30:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.527+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4eb30133:::obj30:head 2026-02-11T23:31:17.527+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4eb30133:::obj30:head v 107'30 2026-02-11T23:31:17.527+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.527+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.527+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4eb30133:::obj30:head, version: 107'30, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4eb30133:::obj30:head@107'30, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.527+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4dd77094:::obj38:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.527+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4dd77094:::obj38:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 114/114 lb 4:4dd77094:::obj38:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.527+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.527+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4dd77094:::obj38:head) v3 2026-02-11T23:31:17.527+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dc3027d:::obj550:head local-lis/les=112/113 n=188 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4dd77094:::obj38:head) 2026-02-11T23:31:17.527+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.527+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4dd77094:::obj38:head) v3 finish latency 0.000015 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.527+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.527+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.527+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x49b8f00 con 0x3e4a000 2026-02-11T23:31:17.527+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4eb30133:::obj30:head)]) -- 0x4e0c600 con 0x3e4a000 2026-02-11T23:31:17.527+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.528+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 426 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.528+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 114 2026-02-11T23:31:17.528+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.528+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 427 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4eb30133:::obj30:head) ==== 1061+0+0 (unknown 0 0 0) 0x5163b80 con 0x3e4a000 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.528+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4eb30133:::obj30:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.528+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4eb30133:::obj30:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 114/112 [PushOp(4:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4eb30133:::obj30:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4fbee9e6:::obj123:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4fbee9e6:::obj123:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4fbee9e6:::obj123:head 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4fbee9e6:::obj123:head v 107'123 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.528+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4fbee9e6:::obj123:head, version: 107'123, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4fbee9e6:::obj123:head@107'123, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4eb30133:::obj30:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.528+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4eb30133:::obj30:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 114/114 lb 4:4eb30133:::obj30:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.528+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.528+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4eb30133:::obj30:head) v3 2026-02-11T23:31:17.528+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4dd77094:::obj38:head local-lis/les=112/113 n=189 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4eb30133:::obj30:head) 2026-02-11T23:31:17.528+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.528+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4eb30133:::obj30:head) v3 finish latency 0.000012 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.528+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.528+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5ca80 con 0x3e4a000 2026-02-11T23:31:17.528+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4fbee9e6:::obj123:head)]) -- 0x4e0c480 con 0x3e4a000 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.528+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 428 ==== MOSDPGPush(4.0 114/112 [PushOp(4:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.528+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 114 2026-02-11T23:31:17.528+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.528+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 429 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4fbee9e6:::obj123:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.528+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4fbee9e6:::obj123:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.528+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4fbee9e6:::obj123:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 114/112 [PushOp(4:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4fbee9e6:::obj123:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:4feb0eaf:::obj556:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:4feb0eaf:::obj556:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:4feb0eaf:::obj556:head 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:4feb0eaf:::obj556:head v 107'556 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.528+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.528+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:4feb0eaf:::obj556:head, version: 107'556, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:4feb0eaf:::obj556:head@107'556, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.528+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4fbee9e6:::obj123:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.528+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4fbee9e6:::obj123:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 114/114 lb 4:4fbee9e6:::obj123:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.528+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.528+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4fbee9e6:::obj123:head) v3 2026-02-11T23:31:17.528+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4eb30133:::obj30:head local-lis/les=112/113 n=190 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4fbee9e6:::obj123:head) 2026-02-11T23:31:17.528+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.528+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4fbee9e6:::obj123:head) v3 finish latency 0.000012 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.528+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.529+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.529+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.529+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4b7d680 con 0x3e4a000 2026-02-11T23:31:17.529+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:4feb0eaf:::obj556:head)]) -- 0x4e0de00 con 0x3e4a000 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.529+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 430 ==== MOSDPGPush(4.0 114/112 [PushOp(4:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.529+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.529+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.529+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.529+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 431 ==== pg_backfill(progress 4.0 e 114/114 lb 4:4feb0eaf:::obj556:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.529+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.529+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:4feb0eaf:::obj556:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.529+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4feb0eaf:::obj556:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.529+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.529+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.529+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 114/112 [PushOp(4:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4feb0eaf:::obj556:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.529+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.529+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.529+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.529+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:50174016:::obj321:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.529+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:50174016:::obj321:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.529+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:50174016:::obj321:head 2026-02-11T23:31:17.529+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:50174016:::obj321:head v 107'321 2026-02-11T23:31:17.529+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.529+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.529+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:50174016:::obj321:head, version: 107'321, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50174016:::obj321:head@107'321, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.529+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:4feb0eaf:::obj556:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.529+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4feb0eaf:::obj556:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 114/114 lb 4:4feb0eaf:::obj556:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.529+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.529+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:4feb0eaf:::obj556:head) v3 2026-02-11T23:31:17.529+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4fbee9e6:::obj123:head local-lis/les=112/113 n=191 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:4feb0eaf:::obj556:head) 2026-02-11T23:31:17.529+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.529+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:4feb0eaf:::obj556:head) v3 finish latency 0.000014 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.529+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.529+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.529+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4b7d800 con 0x3e4a000 2026-02-11T23:31:17.529+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:50174016:::obj321:head)]) -- 0x4e0dc80 con 0x3e4a000 2026-02-11T23:31:17.529+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.530+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 432 ==== MOSDPGPush(4.0 114/112 [PushOp(4:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a21a0 con 0x3e4a000 2026-02-11T23:31:17.530+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.530+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.530+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 433 ==== pg_backfill(progress 4.0 e 114/114 lb 4:50174016:::obj321:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.530+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:50174016:::obj321:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.530+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:50174016:::obj321:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 114/112 [PushOp(4:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:50174016:::obj321:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:502bc4ef:::obj531:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:502bc4ef:::obj531:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:502bc4ef:::obj531:head 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:502bc4ef:::obj531:head v 107'531 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.530+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:502bc4ef:::obj531:head, version: 107'531, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:502bc4ef:::obj531:head@107'531, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:50174016:::obj321:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.530+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:50174016:::obj321:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 114/114 lb 4:50174016:::obj321:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.530+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.530+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:50174016:::obj321:head) v3 2026-02-11T23:31:17.530+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:4feb0eaf:::obj556:head local-lis/les=112/113 n=192 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:50174016:::obj321:head) 2026-02-11T23:31:17.530+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.530+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:50174016:::obj321:head) v3 finish latency 0.000015 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.530+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.530+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0c180 con 0x3e4a000 2026-02-11T23:31:17.530+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:502bc4ef:::obj531:head)]) -- 0x4e0d800 con 0x3e4a000 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.530+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 434 ==== MOSDPGPush(4.0 114/112 [PushOp(4:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2680 con 0x3e4a000 2026-02-11T23:31:17.530+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.530+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.530+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 435 ==== pg_backfill(progress 4.0 e 114/114 lb 4:502bc4ef:::obj531:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.530+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:502bc4ef:::obj531:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.530+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:502bc4ef:::obj531:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:502bc4ef:::obj531:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:506f5ba0:::obj253:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:506f5ba0:::obj253:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:506f5ba0:::obj253:head 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:506f5ba0:::obj253:head v 107'253 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.530+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.530+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:506f5ba0:::obj253:head, version: 107'253, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:506f5ba0:::obj253:head@107'253, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.530+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:502bc4ef:::obj531:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.530+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:502bc4ef:::obj531:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 114/114 lb 4:502bc4ef:::obj531:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.530+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.530+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:502bc4ef:::obj531:head) v3 2026-02-11T23:31:17.530+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50174016:::obj321:head local-lis/les=112/113 n=193 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:502bc4ef:::obj531:head) 2026-02-11T23:31:17.530+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.530+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:502bc4ef:::obj531:head) v3 finish latency 0.000025 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.530+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.531+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.531+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.531+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5cd80 con 0x3e4a000 2026-02-11T23:31:17.531+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:506f5ba0:::obj253:head)]) -- 0x4e0d680 con 0x3e4a000 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.531+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 436 ==== MOSDPGPush(4.0 114/112 [PushOp(4:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2b60 con 0x3e4a000 2026-02-11T23:31:17.531+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.531+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.531+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.531+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 437 ==== pg_backfill(progress 4.0 e 114/114 lb 4:506f5ba0:::obj253:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.531+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:506f5ba0:::obj253:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.531+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.531+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:506f5ba0:::obj253:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.531+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.531+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.531+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 114/112 [PushOp(4:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:506f5ba0:::obj253:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.531+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.531+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.531+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.531+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:50fb58bc:::obj174:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.531+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:50fb58bc:::obj174:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.531+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:50fb58bc:::obj174:head 2026-02-11T23:31:17.531+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:50fb58bc:::obj174:head v 107'174 2026-02-11T23:31:17.531+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.531+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.531+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:50fb58bc:::obj174:head, version: 107'174, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:50fb58bc:::obj174:head@107'174, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.531+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:506f5ba0:::obj253:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.531+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:506f5ba0:::obj253:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 114/114 lb 4:506f5ba0:::obj253:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.531+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.531+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:506f5ba0:::obj253:head) v3 2026-02-11T23:31:17.531+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:502bc4ef:::obj531:head local-lis/les=112/113 n=194 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:506f5ba0:::obj253:head) 2026-02-11T23:31:17.531+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.531+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:506f5ba0:::obj253:head) v3 finish latency 0.000017 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.531+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.531+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.531+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5dc80 con 0x3e4a000 2026-02-11T23:31:17.531+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:50fb58bc:::obj174:head)]) -- 0x4e0d500 con 0x3e4a000 2026-02-11T23:31:17.531+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.532+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 438 ==== MOSDPGPush(4.0 114/112 [PushOp(4:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dfe1a0 con 0x3e4a000 2026-02-11T23:31:17.532+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.532+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.532+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 439 ==== pg_backfill(progress 4.0 e 114/114 lb 4:50fb58bc:::obj174:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.532+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:50fb58bc:::obj174:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.532+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:50fb58bc:::obj174:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:50fb58bc:::obj174:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:51058f37:::obj394:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:51058f37:::obj394:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:51058f37:::obj394:head 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:51058f37:::obj394:head v 107'394 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.532+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:51058f37:::obj394:head, version: 107'394, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:51058f37:::obj394:head@107'394, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:50fb58bc:::obj174:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:50fb58bc:::obj174:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 114/114 lb 4:50fb58bc:::obj174:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:50fb58bc:::obj174:head) v3 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:506f5ba0:::obj253:head local-lis/les=112/113 n=195 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:50fb58bc:::obj174:head) 2026-02-11T23:31:17.532+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:50fb58bc:::obj174:head) v3 finish latency 0.000014 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.532+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c97200 con 0x3e4a000 2026-02-11T23:31:17.532+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:51058f37:::obj394:head)]) -- 0x4e0d380 con 0x3e4a000 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.532+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 440 ==== MOSDPGPush(4.0 114/112 [PushOp(4:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3520 con 0x3e4a000 2026-02-11T23:31:17.532+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.532+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.532+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 441 ==== pg_backfill(progress 4.0 e 114/114 lb 4:51058f37:::obj394:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.532+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:51058f37:::obj394:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.532+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:51058f37:::obj394:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:51058f37:::obj394:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 114/112 [PushOp(4:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:5170da72:::obj410:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:5170da72:::obj410:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:5170da72:::obj410:head 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:5170da72:::obj410:head v 107'410 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.532+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.532+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5170da72:::obj410:head, version: 107'410, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5170da72:::obj410:head@107'410, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:51058f37:::obj394:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:51058f37:::obj394:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 114/114 lb 4:51058f37:::obj394:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:51058f37:::obj394:head) v3 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:50fb58bc:::obj174:head local-lis/les=112/113 n=196 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:51058f37:::obj394:head) 2026-02-11T23:31:17.532+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:51058f37:::obj394:head) v3 finish latency 0.000016 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.532+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.532+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.532+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8fe00 con 0x3e4a000 2026-02-11T23:31:17.532+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:5170da72:::obj410:head)]) -- 0x4e0d200 con 0x3e4a000 2026-02-11T23:31:17.532+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.533+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 442 ==== MOSDPGPush(4.0 114/112 [PushOp(4:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183d40 con 0x3e4a000 2026-02-11T23:31:17.533+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.533+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.533+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 443 ==== pg_backfill(progress 4.0 e 114/114 lb 4:5170da72:::obj410:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5600 con 0x3e4a000 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.533+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:5170da72:::obj410:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.533+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5170da72:::obj410:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 114/112 [PushOp(4:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5170da72:::obj410:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:5178a392:::obj281:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:5178a392:::obj281:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:5178a392:::obj281:head 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:5178a392:::obj281:head v 107'281 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.533+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5178a392:::obj281:head, version: 107'281, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5178a392:::obj281:head@107'281, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5170da72:::obj410:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.533+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5170da72:::obj410:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 114/114 lb 4:5170da72:::obj410:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.533+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.533+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:5170da72:::obj410:head) v3 2026-02-11T23:31:17.533+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:51058f37:::obj394:head local-lis/les=112/113 n=197 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:5170da72:::obj410:head) 2026-02-11T23:31:17.533+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.533+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5170da72:::obj410:head) v3 finish latency 0.000039 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.533+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.533+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5cf00 con 0x3e4a000 2026-02-11T23:31:17.533+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:5178a392:::obj281:head)]) -- 0x4e0d080 con 0x3e4a000 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.533+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 444 ==== MOSDPGPush(4.0 114/112 [PushOp(4:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3e4a000 2026-02-11T23:31:17.533+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.533+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.533+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 445 ==== pg_backfill(progress 4.0 e 114/114 lb 4:5178a392:::obj281:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.533+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:5178a392:::obj281:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.533+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5178a392:::obj281:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5178a392:::obj281:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:52d1d495:::obj578:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:52d1d495:::obj578:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:52d1d495:::obj578:head 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:52d1d495:::obj578:head v 107'578 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.533+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.533+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:52d1d495:::obj578:head, version: 107'578, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52d1d495:::obj578:head@107'578, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.533+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5178a392:::obj281:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.533+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5178a392:::obj281:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 114/114 lb 4:5178a392:::obj281:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.533+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.533+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:5178a392:::obj281:head) v3 2026-02-11T23:31:17.533+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5170da72:::obj410:head local-lis/les=112/113 n=198 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:5178a392:::obj281:head) 2026-02-11T23:31:17.533+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.533+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5178a392:::obj281:head) v3 finish latency 0.000022 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.533+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.534+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.534+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.534+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8fb00 con 0x3e4a000 2026-02-11T23:31:17.534+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:52d1d495:::obj578:head)]) -- 0x4e0cf00 con 0x3e4a000 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.534+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 446 ==== MOSDPGPush(4.0 114/112 [PushOp(4:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.534+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.534+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.534+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.534+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 447 ==== pg_backfill(progress 4.0 e 114/114 lb 4:52d1d495:::obj578:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3e4a000 2026-02-11T23:31:17.534+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.534+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:52d1d495:::obj578:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.534+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:52d1d495:::obj578:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.534+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.534+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.534+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 114/112 [PushOp(4:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.534+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.534+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.534+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:52d1d495:::obj578:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.534+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:52e063dd:::obj399:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.534+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:52e063dd:::obj399:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.534+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:52e063dd:::obj399:head 2026-02-11T23:31:17.534+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:52e063dd:::obj399:head v 107'399 2026-02-11T23:31:17.534+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.534+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.534+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:52e063dd:::obj399:head, version: 107'399, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:52e063dd:::obj399:head@107'399, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.534+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:52d1d495:::obj578:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.534+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:52d1d495:::obj578:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 114/114 lb 4:52d1d495:::obj578:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.534+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.534+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:52d1d495:::obj578:head) v3 2026-02-11T23:31:17.534+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5178a392:::obj281:head local-lis/les=112/113 n=199 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:52d1d495:::obj578:head) 2026-02-11T23:31:17.534+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.534+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:52d1d495:::obj578:head) v3 finish latency 0.000012 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.534+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.534+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.534+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4b7dc80 con 0x3e4a000 2026-02-11T23:31:17.534+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:52e063dd:::obj399:head)]) -- 0x4e0cd80 con 0x3e4a000 2026-02-11T23:31:17.534+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.535+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 448 ==== MOSDPGPush(4.0 114/112 [PushOp(4:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.535+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.535+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.535+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 449 ==== pg_backfill(progress 4.0 e 114/114 lb 4:52e063dd:::obj399:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5080 con 0x3e4a000 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.535+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:52e063dd:::obj399:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.535+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:52e063dd:::obj399:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:52e063dd:::obj399:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:53438dcf:::obj478:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:53438dcf:::obj478:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:53438dcf:::obj478:head 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:53438dcf:::obj478:head v 107'478 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.535+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:53438dcf:::obj478:head, version: 107'478, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53438dcf:::obj478:head@107'478, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:52e063dd:::obj399:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:52e063dd:::obj399:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 114/114 lb 4:52e063dd:::obj399:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:52e063dd:::obj399:head) v3 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52d1d495:::obj578:head local-lis/les=112/113 n=200 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:52e063dd:::obj399:head) 2026-02-11T23:31:17.535+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:52e063dd:::obj399:head) v3 finish latency 0.000010 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.535+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.535+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:53438dcf:::obj478:head)]) -- 0x4e0c900 con 0x3e4a000 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.535+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 450 ==== MOSDPGPush(4.0 114/112 [PushOp(4:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.535+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.535+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.535+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 451 ==== pg_backfill(progress 4.0 e 114/114 lb 4:53438dcf:::obj478:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3e4a000 2026-02-11T23:31:17.535+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:53438dcf:::obj478:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.535+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:53438dcf:::obj478:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:53438dcf:::obj478:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:53c971fd:::obj132:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:53c971fd:::obj132:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:53c971fd:::obj132:head 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:53c971fd:::obj132:head v 107'132 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.535+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.535+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:53c971fd:::obj132:head, version: 107'132, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:53c971fd:::obj132:head@107'132, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:53438dcf:::obj478:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:53438dcf:::obj478:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:53438dcf:::obj478:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:53438dcf:::obj478:head) v3 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:52e063dd:::obj399:head local-lis/les=112/113 n=201 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:53438dcf:::obj478:head) 2026-02-11T23:31:17.535+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:53438dcf:::obj478:head) v3 finish latency 0.000011 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.535+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.535+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.535+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d080 con 0x3e4a000 2026-02-11T23:31:17.535+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:53c971fd:::obj132:head)]) -- 0x4c1e000 con 0x3e4a000 2026-02-11T23:31:17.535+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.536+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 452 ==== MOSDPGPush(4.0 114/112 [PushOp(4:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.536+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.536+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.536+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 453 ==== pg_backfill(progress 4.0 e 114/114 lb 4:53c971fd:::obj132:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3e4a000 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.536+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:53c971fd:::obj132:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.536+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:53c971fd:::obj132:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:53c971fd:::obj132:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:541e28f4:::obj33:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:541e28f4:::obj33:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:541e28f4:::obj33:head 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:541e28f4:::obj33:head v 107'33 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.536+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:541e28f4:::obj33:head, version: 107'33, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:541e28f4:::obj33:head@107'33, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:53c971fd:::obj132:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:53c971fd:::obj132:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 114/114 lb 4:53c971fd:::obj132:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:53c971fd:::obj132:head) v3 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53438dcf:::obj478:head local-lis/les=112/113 n=202 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:53c971fd:::obj132:head) 2026-02-11T23:31:17.536+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:53c971fd:::obj132:head) v3 finish latency 0.000013 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.536+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8f500 con 0x3e4a000 2026-02-11T23:31:17.536+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:541e28f4:::obj33:head)]) -- 0x4c1e300 con 0x3e4a000 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.536+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 454 ==== MOSDPGPush(4.0 114/112 [PushOp(4:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3e4a000 2026-02-11T23:31:17.536+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.536+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.536+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 455 ==== pg_backfill(progress 4.0 e 114/114 lb 4:541e28f4:::obj33:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3e4a000 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.536+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:541e28f4:::obj33:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.536+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:541e28f4:::obj33:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:541e28f4:::obj33:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:545db4bf:::obj139:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:545db4bf:::obj139:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:545db4bf:::obj139:head 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:545db4bf:::obj139:head v 107'139 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.536+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.536+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:545db4bf:::obj139:head, version: 107'139, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:545db4bf:::obj139:head@107'139, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:541e28f4:::obj33:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:541e28f4:::obj33:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:541e28f4:::obj33:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:541e28f4:::obj33:head) v3 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:53c971fd:::obj132:head local-lis/les=112/113 n=203 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:541e28f4:::obj33:head) 2026-02-11T23:31:17.536+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:541e28f4:::obj33:head) v3 finish latency 0.000012 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.536+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.536+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.536+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.537+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x49b8c00 con 0x3e4a000 2026-02-11T23:31:17.537+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:545db4bf:::obj139:head)]) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.537+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 456 ==== MOSDPGPush(4.0 114/112 [PushOp(4:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3e4a000 2026-02-11T23:31:17.537+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.537+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.537+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.537+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 457 ==== pg_backfill(progress 4.0 e 114/114 lb 4:545db4bf:::obj139:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3e4a000 2026-02-11T23:31:17.537+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.537+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:545db4bf:::obj139:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.537+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:545db4bf:::obj139:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.537+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.537+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.537+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.537+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.537+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:545db4bf:::obj139:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.537+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.537+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:54a9e115:::obj404:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.537+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:54a9e115:::obj404:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.537+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:54a9e115:::obj404:head 2026-02-11T23:31:17.537+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:54a9e115:::obj404:head v 107'404 2026-02-11T23:31:17.537+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.537+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.537+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:54a9e115:::obj404:head, version: 107'404, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54a9e115:::obj404:head@107'404, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.537+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:545db4bf:::obj139:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.537+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:545db4bf:::obj139:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:545db4bf:::obj139:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.537+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.537+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:545db4bf:::obj139:head) v3 2026-02-11T23:31:17.537+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:541e28f4:::obj33:head local-lis/les=112/113 n=204 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:545db4bf:::obj139:head) 2026-02-11T23:31:17.537+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.537+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:545db4bf:::obj139:head) v3 finish latency 0.000012 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.537+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.537+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.537+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8f200 con 0x3e4a000 2026-02-11T23:31:17.537+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:54a9e115:::obj404:head)]) -- 0x4c1e780 con 0x3e4a000 2026-02-11T23:31:17.537+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.538+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 458 ==== MOSDPGPush(4.0 114/112 [PushOp(4:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51821a0 con 0x3e4a000 2026-02-11T23:31:17.538+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.538+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.538+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 459 ==== pg_backfill(progress 4.0 e 114/114 lb 4:54a9e115:::obj404:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3e4a000 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.538+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:54a9e115:::obj404:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.538+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:54a9e115:::obj404:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:54a9e115:::obj404:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:54cd4daa:::obj313:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:54cd4daa:::obj313:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:54cd4daa:::obj313:head 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:54cd4daa:::obj313:head v 107'313 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.538+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:54cd4daa:::obj313:head, version: 107'313, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:54cd4daa:::obj313:head@107'313, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:54a9e115:::obj404:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:54a9e115:::obj404:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:54a9e115:::obj404:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:54a9e115:::obj404:head) v3 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:545db4bf:::obj139:head local-lis/les=112/113 n=205 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:54a9e115:::obj404:head) 2026-02-11T23:31:17.538+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:54a9e115:::obj404:head) v3 finish latency 0.000009 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.538+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.538+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:54cd4daa:::obj313:head)]) -- 0x4c1ea80 con 0x3e4a000 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.538+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 460 ==== MOSDPGPush(4.0 114/112 [PushOp(4:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3e4a000 2026-02-11T23:31:17.538+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.538+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.538+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 461 ==== pg_backfill(progress 4.0 e 114/114 lb 4:54cd4daa:::obj313:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3e4a000 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.538+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:54cd4daa:::obj313:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.538+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:54cd4daa:::obj313:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:54cd4daa:::obj313:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:554d3990:::obj276:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:554d3990:::obj276:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:554d3990:::obj276:head 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:554d3990:::obj276:head v 107'276 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.538+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.538+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:554d3990:::obj276:head, version: 107'276, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:554d3990:::obj276:head@107'276, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:54cd4daa:::obj313:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:54cd4daa:::obj313:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 114/114 lb 4:54cd4daa:::obj313:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:54cd4daa:::obj313:head) v3 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54a9e115:::obj404:head local-lis/les=112/113 n=206 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:54cd4daa:::obj313:head) 2026-02-11T23:31:17.538+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:54cd4daa:::obj313:head) v3 finish latency 0.000011 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.538+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.538+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.538+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8ef00 con 0x3e4a000 2026-02-11T23:31:17.538+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:554d3990:::obj276:head)]) -- 0x4c1ed80 con 0x3e4a000 2026-02-11T23:31:17.538+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.539+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 462 ==== MOSDPGPush(4.0 114/112 [PushOp(4:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51831e0 con 0x3e4a000 2026-02-11T23:31:17.539+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.539+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.539+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 463 ==== pg_backfill(progress 4.0 e 114/114 lb 4:554d3990:::obj276:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3e4a000 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.539+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:554d3990:::obj276:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.539+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:554d3990:::obj276:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:554d3990:::obj276:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:55c8626c:::obj516:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:55c8626c:::obj516:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:55c8626c:::obj516:head 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:55c8626c:::obj516:head v 107'516 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.539+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:55c8626c:::obj516:head, version: 107'516, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55c8626c:::obj516:head@107'516, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:554d3990:::obj276:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.539+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:554d3990:::obj276:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 114/114 lb 4:554d3990:::obj276:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.539+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.539+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:554d3990:::obj276:head) v3 2026-02-11T23:31:17.539+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:54cd4daa:::obj313:head local-lis/les=112/113 n=207 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:554d3990:::obj276:head) 2026-02-11T23:31:17.539+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.539+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:554d3990:::obj276:head) v3 finish latency 0.000014 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.539+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.539+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.539+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:55c8626c:::obj516:head)]) -- 0x4c1f080 con 0x3e4a000 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.539+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 464 ==== MOSDPGPush(4.0 114/112 [PushOp(4:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3e4a000 2026-02-11T23:31:17.539+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.539+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.539+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 465 ==== pg_backfill(progress 4.0 e 114/114 lb 4:55c8626c:::obj516:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3e4a000 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.539+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:55c8626c:::obj516:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.539+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:55c8626c:::obj516:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:55c8626c:::obj516:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:55d2244d:::obj498:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:55d2244d:::obj498:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:55d2244d:::obj498:head 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:55d2244d:::obj498:head v 107'498 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.539+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.539+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:55d2244d:::obj498:head, version: 107'498, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:55d2244d:::obj498:head@107'498, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.539+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:55c8626c:::obj516:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.539+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:55c8626c:::obj516:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:55c8626c:::obj516:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.539+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.539+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:55c8626c:::obj516:head) v3 2026-02-11T23:31:17.539+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:554d3990:::obj276:head local-lis/les=112/113 n=208 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:55c8626c:::obj516:head) 2026-02-11T23:31:17.539+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.539+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:55c8626c:::obj516:head) v3 finish latency 0.000012 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.539+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.540+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.540+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.540+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x49d7800 con 0x3e4a000 2026-02-11T23:31:17.540+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:55d2244d:::obj498:head)]) -- 0x4c1f380 con 0x3e4a000 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.540+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 466 ==== MOSDPGPush(4.0 114/112 [PushOp(4:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.540+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.540+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.540+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.540+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 467 ==== pg_backfill(progress 4.0 e 114/114 lb 4:55d2244d:::obj498:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3e4a000 2026-02-11T23:31:17.540+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:55d2244d:::obj498:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 114 2026-02-11T23:31:17.540+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.540+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:55d2244d:::obj498:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.540+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.540+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:55d2244d:::obj498:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.540+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 114/112 [PushOp(4:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.540+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.540+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.540+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.540+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:565a3fbf:::obj474:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.540+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:565a3fbf:::obj474:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.540+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:565a3fbf:::obj474:head 2026-02-11T23:31:17.540+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:565a3fbf:::obj474:head v 107'474 2026-02-11T23:31:17.540+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.540+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.540+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:565a3fbf:::obj474:head, version: 107'474, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:565a3fbf:::obj474:head@107'474, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.540+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:55d2244d:::obj498:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.540+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:55d2244d:::obj498:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 114/114 lb 4:55d2244d:::obj498:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.540+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.540+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:55d2244d:::obj498:head) v3 2026-02-11T23:31:17.540+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55c8626c:::obj516:head local-lis/les=112/113 n=209 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:55d2244d:::obj498:head) 2026-02-11T23:31:17.540+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.540+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:55d2244d:::obj498:head) v3 finish latency 0.000012 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.540+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.540+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.540+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d380 con 0x3e4a000 2026-02-11T23:31:17.540+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:565a3fbf:::obj474:head)]) -- 0x4c1f680 con 0x3e4a000 2026-02-11T23:31:17.540+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.541+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 468 ==== MOSDPGPush(4.0 114/112 [PushOp(4:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.541+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.541+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.541+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 469 ==== pg_backfill(progress 4.0 e 114/114 lb 4:565a3fbf:::obj474:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3e4a000 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.541+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:565a3fbf:::obj474:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.541+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:565a3fbf:::obj474:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:565a3fbf:::obj474:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:5695f913:::obj241:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:5695f913:::obj241:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:5695f913:::obj241:head 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:5695f913:::obj241:head v 107'241 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.541+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5695f913:::obj241:head, version: 107'241, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5695f913:::obj241:head@107'241, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:565a3fbf:::obj474:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:565a3fbf:::obj474:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:565a3fbf:::obj474:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:565a3fbf:::obj474:head) v3 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:55d2244d:::obj498:head local-lis/les=112/113 n=210 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:565a3fbf:::obj474:head) 2026-02-11T23:31:17.541+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:565a3fbf:::obj474:head) v3 finish latency 0.000011 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.541+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8ec00 con 0x3e4a000 2026-02-11T23:31:17.541+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:5695f913:::obj241:head)]) -- 0x4c1f980 con 0x3e4a000 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.541+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 470 ==== MOSDPGPush(4.0 114/112 [PushOp(4:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.541+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.541+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.541+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 471 ==== pg_backfill(progress 4.0 e 114/114 lb 4:5695f913:::obj241:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded080 con 0x3e4a000 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.541+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:5695f913:::obj241:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.541+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5695f913:::obj241:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5695f913:::obj241:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:56fc6216:::obj287:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:56fc6216:::obj287:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:56fc6216:::obj287:head 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:56fc6216:::obj287:head v 107'287 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.541+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.541+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:56fc6216:::obj287:head, version: 107'287, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:56fc6216:::obj287:head@107'287, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5695f913:::obj241:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5695f913:::obj241:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:5695f913:::obj241:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:5695f913:::obj241:head) v3 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:565a3fbf:::obj474:head local-lis/les=112/113 n=211 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:5695f913:::obj241:head) 2026-02-11T23:31:17.541+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5695f913:::obj241:head) v3 finish latency 0.000014 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.541+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.541+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.541+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d680 con 0x3e4a000 2026-02-11T23:31:17.541+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:56fc6216:::obj287:head)]) -- 0x4c1fc80 con 0x3e4a000 2026-02-11T23:31:17.541+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.542+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 472 ==== MOSDPGPush(4.0 114/112 [PushOp(4:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.542+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.542+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.542+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 473 ==== pg_backfill(progress 4.0 e 114/114 lb 4:56fc6216:::obj287:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3e4a000 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.542+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:56fc6216:::obj287:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.542+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:56fc6216:::obj287:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:56fc6216:::obj287:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:578304e3:::obj249:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:578304e3:::obj249:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:578304e3:::obj249:head 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:578304e3:::obj249:head v 107'249 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.542+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:578304e3:::obj249:head, version: 107'249, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:578304e3:::obj249:head@107'249, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:56fc6216:::obj287:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.542+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:56fc6216:::obj287:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 114/114 lb 4:56fc6216:::obj287:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.542+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.542+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:56fc6216:::obj287:head) v3 2026-02-11T23:31:17.542+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5695f913:::obj241:head local-lis/les=112/113 n=212 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:56fc6216:::obj287:head) 2026-02-11T23:31:17.542+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.542+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:56fc6216:::obj287:head) v3 finish latency 0.000012 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.542+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.542+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4fcf200 con 0x3e4a000 2026-02-11T23:31:17.542+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:578304e3:::obj249:head)]) -- 0x4d8f500 con 0x3e4a000 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.542+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 474 ==== MOSDPGPush(4.0 114/112 [PushOp(4:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3e4a000 2026-02-11T23:31:17.542+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.542+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.542+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 475 ==== pg_backfill(progress 4.0 e 114/114 lb 4:578304e3:::obj249:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3e4a000 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.542+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:578304e3:::obj249:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.542+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:578304e3:::obj249:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:578304e3:::obj249:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:57ba9973:::obj296:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:57ba9973:::obj296:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:57ba9973:::obj296:head 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:57ba9973:::obj296:head v 107'296 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.542+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.542+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:57ba9973:::obj296:head, version: 107'296, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:57ba9973:::obj296:head@107'296, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.542+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.542+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:578304e3:::obj249:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.542+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:578304e3:::obj249:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:578304e3:::obj249:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.542+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.542+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:578304e3:::obj249:head) v3 2026-02-11T23:31:17.542+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:56fc6216:::obj287:head local-lis/les=112/113 n=213 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:578304e3:::obj249:head) 2026-02-11T23:31:17.543+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.543+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:578304e3:::obj249:head) v3 finish latency 0.000014 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.543+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.543+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.543+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d500 con 0x3e4a000 2026-02-11T23:31:17.543+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:57ba9973:::obj296:head)]) -- 0x4e5d080 con 0x3e4a000 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.543+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 476 ==== MOSDPGPush(4.0 114/112 [PushOp(4:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183d40 con 0x3e4a000 2026-02-11T23:31:17.543+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.543+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.543+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.543+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.543+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 477 ==== pg_backfill(progress 4.0 e 114/114 lb 4:57ba9973:::obj296:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3e4a000 2026-02-11T23:31:17.543+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.543+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:57ba9973:::obj296:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 114 2026-02-11T23:31:17.543+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.543+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:57ba9973:::obj296:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.543+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 114/112 [PushOp(4:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:57ba9973:::obj296:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.543+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.543+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.543+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.543+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:5880e7a7:::obj64:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.543+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:5880e7a7:::obj64:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.543+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:5880e7a7:::obj64:head 2026-02-11T23:31:17.543+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:5880e7a7:::obj64:head v 107'64 2026-02-11T23:31:17.543+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.543+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.543+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5880e7a7:::obj64:head, version: 107'64, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5880e7a7:::obj64:head@107'64, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.543+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:57ba9973:::obj296:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.543+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:57ba9973:::obj296:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:57ba9973:::obj296:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.543+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.543+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:57ba9973:::obj296:head) v3 2026-02-11T23:31:17.543+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:578304e3:::obj249:head local-lis/les=112/113 n=214 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:57ba9973:::obj296:head) 2026-02-11T23:31:17.543+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.543+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:57ba9973:::obj296:head) v3 finish latency 0.000010 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.543+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.543+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.543+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.543+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:5880e7a7:::obj64:head)]) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.543+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.544+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 478 ==== MOSDPGPush(4.0 114/112 [PushOp(4:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3520 con 0x3e4a000 2026-02-11T23:31:17.544+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.544+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.544+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 479 ==== pg_backfill(progress 4.0 e 114/114 lb 4:5880e7a7:::obj64:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3e4a000 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.544+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:5880e7a7:::obj64:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.544+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5880e7a7:::obj64:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5880e7a7:::obj64:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:58980d3a:::obj339:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:58980d3a:::obj339:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:58980d3a:::obj339:head 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:58980d3a:::obj339:head v 107'339 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.544+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:58980d3a:::obj339:head, version: 107'339, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:58980d3a:::obj339:head@107'339, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5880e7a7:::obj64:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5880e7a7:::obj64:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 114/114 lb 4:5880e7a7:::obj64:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:5880e7a7:::obj64:head) v3 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:57ba9973:::obj296:head local-lis/les=112/113 n=215 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:5880e7a7:::obj64:head) 2026-02-11T23:31:17.544+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5880e7a7:::obj64:head) v3 finish latency 0.000010 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.544+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d980 con 0x3e4a000 2026-02-11T23:31:17.544+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:58980d3a:::obj339:head)]) -- 0x4e0cd80 con 0x3e4a000 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.544+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 480 ==== MOSDPGPush(4.0 114/112 [PushOp(4:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dfe1a0 con 0x3e4a000 2026-02-11T23:31:17.544+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.544+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.544+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 481 ==== pg_backfill(progress 4.0 e 114/114 lb 4:58980d3a:::obj339:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd180 con 0x3e4a000 2026-02-11T23:31:17.544+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:58980d3a:::obj339:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.544+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:58980d3a:::obj339:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:58980d3a:::obj339:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:5946d48f:::obj257:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:5946d48f:::obj257:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:5946d48f:::obj257:head 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:5946d48f:::obj257:head v 107'257 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.544+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.544+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5946d48f:::obj257:head, version: 107'257, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5946d48f:::obj257:head@107'257, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:58980d3a:::obj339:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:58980d3a:::obj339:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:58980d3a:::obj339:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:58980d3a:::obj339:head) v3 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5880e7a7:::obj64:head local-lis/les=112/113 n=216 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:58980d3a:::obj339:head) 2026-02-11T23:31:17.544+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:58980d3a:::obj339:head) v3 finish latency 0.000010 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.544+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.544+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.544+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8e900 con 0x3e4a000 2026-02-11T23:31:17.544+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:5946d48f:::obj257:head)]) -- 0x4e0cf00 con 0x3e4a000 2026-02-11T23:31:17.544+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.545+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 482 ==== MOSDPGPush(4.0 114/112 [PushOp(4:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2b60 con 0x3e4a000 2026-02-11T23:31:17.545+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.545+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.545+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 483 ==== pg_backfill(progress 4.0 e 114/114 lb 4:5946d48f:::obj257:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def180 con 0x3e4a000 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.545+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:5946d48f:::obj257:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 114 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.545+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5946d48f:::obj257:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5946d48f:::obj257:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 114/112 [PushOp(4:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:5a01aba7:::obj211:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:5a01aba7:::obj211:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:5a01aba7:::obj211:head 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:5a01aba7:::obj211:head v 107'211 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.545+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5a01aba7:::obj211:head, version: 107'211, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a01aba7:::obj211:head@107'211, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5946d48f:::obj257:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.545+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5946d48f:::obj257:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 114/114 lb 4:5946d48f:::obj257:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.545+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.545+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:5946d48f:::obj257:head) v3 2026-02-11T23:31:17.545+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:58980d3a:::obj339:head local-lis/les=112/113 n=217 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:5946d48f:::obj257:head) 2026-02-11T23:31:17.545+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.545+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5946d48f:::obj257:head) v3 finish latency 0.000012 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.545+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.545+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5db00 con 0x3e4a000 2026-02-11T23:31:17.545+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:5a01aba7:::obj211:head)]) -- 0x4e0d080 con 0x3e4a000 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.545+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 484 ==== MOSDPGPush(4.0 114/112 [PushOp(4:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2680 con 0x3e4a000 2026-02-11T23:31:17.545+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 114 2026-02-11T23:31:17.545+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.545+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 485 ==== pg_backfill(progress 4.0 e 114/114 lb 4:5a01aba7:::obj211:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3e4a000 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.545+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:5a01aba7:::obj211:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.545+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5a01aba7:::obj211:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5a01aba7:::obj211:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:5a322cd1:::obj402:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:5a322cd1:::obj402:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:5a322cd1:::obj402:head 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:5a322cd1:::obj402:head v 107'402 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.545+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.545+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5a322cd1:::obj402:head, version: 107'402, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a322cd1:::obj402:head@107'402, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.545+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5a01aba7:::obj211:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.545+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5a01aba7:::obj211:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:5a01aba7:::obj211:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.545+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.545+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:5a01aba7:::obj211:head) v3 2026-02-11T23:31:17.545+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5946d48f:::obj257:head local-lis/les=112/113 n=218 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:5a01aba7:::obj211:head) 2026-02-11T23:31:17.545+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.545+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5a01aba7:::obj211:head) v3 finish latency 0.000009 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.545+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.546+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.546+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.546+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8e780 con 0x3e4a000 2026-02-11T23:31:17.546+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:5a322cd1:::obj402:head)]) -- 0x4e0d200 con 0x3e4a000 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.546+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 486 ==== MOSDPGPush(4.0 114/112 [PushOp(4:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a21a0 con 0x3e4a000 2026-02-11T23:31:17.546+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.546+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.546+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.546+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 487 ==== pg_backfill(progress 4.0 e 114/114 lb 4:5a322cd1:::obj402:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164c00 con 0x3e4a000 2026-02-11T23:31:17.546+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:5a322cd1:::obj402:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.546+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.546+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5a322cd1:::obj402:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.546+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.546+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.546+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 114/112 [PushOp(4:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.546+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.546+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.546+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5a322cd1:::obj402:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.546+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:5a3c6224:::obj352:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.546+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:5a3c6224:::obj352:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.546+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:5a3c6224:::obj352:head 2026-02-11T23:31:17.546+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:5a3c6224:::obj352:head v 107'352 2026-02-11T23:31:17.546+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.546+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.546+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5a3c6224:::obj352:head, version: 107'352, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5a3c6224:::obj352:head@107'352, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.546+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5a322cd1:::obj402:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.546+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5a322cd1:::obj402:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 114/114 lb 4:5a322cd1:::obj402:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.546+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.546+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:5a322cd1:::obj402:head) v3 2026-02-11T23:31:17.546+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a01aba7:::obj211:head local-lis/les=112/113 n=219 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:5a322cd1:::obj402:head) 2026-02-11T23:31:17.546+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.546+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5a322cd1:::obj402:head) v3 finish latency 0.000010 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.546+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.546+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.546+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4b7d500 con 0x3e4a000 2026-02-11T23:31:17.546+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:5a3c6224:::obj352:head)]) -- 0x4e0d380 con 0x3e4a000 2026-02-11T23:31:17.546+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.547+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 488 ==== MOSDPGPush(4.0 114/112 [PushOp(4:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.547+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.547+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.547+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 489 ==== pg_backfill(progress 4.0 e 114/114 lb 4:5a3c6224:::obj352:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165180 con 0x3e4a000 2026-02-11T23:31:17.547+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:5a3c6224:::obj352:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.547+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5a3c6224:::obj352:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5a3c6224:::obj352:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000036 MOSDPGPush(4.0 114/112 [PushOp(4:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:5d17b8a1:::obj341:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:5d17b8a1:::obj341:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:5d17b8a1:::obj341:head 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:5d17b8a1:::obj341:head v 107'341 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.547+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5d17b8a1:::obj341:head, version: 107'341, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5d17b8a1:::obj341:head@107'341, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5a3c6224:::obj352:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5a3c6224:::obj352:head) v3 prio 15 cost 0 latency 0.000100 pg_backfill(progress 4.0 e 114/114 lb 4:5a3c6224:::obj352:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:5a3c6224:::obj352:head) v3 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a322cd1:::obj402:head local-lis/les=112/113 n=220 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:5a3c6224:::obj352:head) 2026-02-11T23:31:17.547+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5a3c6224:::obj352:head) v3 finish latency 0.000013 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.547+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8e180 con 0x3e4a000 2026-02-11T23:31:17.547+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:5d17b8a1:::obj341:head)]) -- 0x4d8e480 con 0x3e4a000 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.547+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 490 ==== MOSDPGPush(4.0 114/112 [PushOp(4:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.547+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.547+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.547+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 491 ==== pg_backfill(progress 4.0 e 114/114 lb 4:5d17b8a1:::obj341:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3e4a000 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.547+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:5d17b8a1:::obj341:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.547+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5d17b8a1:::obj341:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5d17b8a1:::obj341:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:5e27c68c:::obj582:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:5e27c68c:::obj582:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:5e27c68c:::obj582:head 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:5e27c68c:::obj582:head v 107'582 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.547+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.547+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:5e27c68c:::obj582:head, version: 107'582, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:5e27c68c:::obj582:head@107'582, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5d17b8a1:::obj341:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5d17b8a1:::obj341:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 114/114 lb 4:5d17b8a1:::obj341:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:5d17b8a1:::obj341:head) v3 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5a3c6224:::obj352:head local-lis/les=112/113 n=221 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:5d17b8a1:::obj341:head) 2026-02-11T23:31:17.547+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5d17b8a1:::obj341:head) v3 finish latency 0.000012 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.547+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.547+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.547+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.548+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8e780 con 0x3e4a000 2026-02-11T23:31:17.548+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:5e27c68c:::obj582:head)]) -- 0x4e0d500 con 0x3e4a000 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.548+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 492 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.548+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.548+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.548+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 493 ==== pg_backfill(progress 4.0 e 114/114 lb 4:5e27c68c:::obj582:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.548+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:5e27c68c:::obj582:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.548+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5e27c68c:::obj582:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5e27c68c:::obj582:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6131e0b1:::obj290:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6131e0b1:::obj290:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6131e0b1:::obj290:head 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6131e0b1:::obj290:head v 107'290 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.548+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6131e0b1:::obj290:head, version: 107'290, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6131e0b1:::obj290:head@107'290, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:5e27c68c:::obj582:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.548+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5e27c68c:::obj582:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 114/114 lb 4:5e27c68c:::obj582:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.548+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.548+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:5e27c68c:::obj582:head) v3 2026-02-11T23:31:17.548+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5d17b8a1:::obj341:head local-lis/les=112/113 n=222 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:5e27c68c:::obj582:head) 2026-02-11T23:31:17.548+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.548+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:5e27c68c:::obj582:head) v3 finish latency 0.000011 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.548+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.548+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0d080 con 0x3e4a000 2026-02-11T23:31:17.548+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6131e0b1:::obj290:head)]) -- 0x4e0d680 con 0x3e4a000 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.548+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 494 ==== MOSDPGPush(4.0 114/112 [PushOp(4:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.548+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.548+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.548+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 495 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6131e0b1:::obj290:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.548+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6131e0b1:::obj290:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 114 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.548+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6131e0b1:::obj290:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6131e0b1:::obj290:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:614752af:::obj331:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:614752af:::obj331:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:614752af:::obj331:head 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:614752af:::obj331:head v 107'331 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.548+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.548+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:614752af:::obj331:head, version: 107'331, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:614752af:::obj331:head@107'331, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.548+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6131e0b1:::obj290:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.548+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6131e0b1:::obj290:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:6131e0b1:::obj290:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.548+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.548+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6131e0b1:::obj290:head) v3 2026-02-11T23:31:17.548+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:5e27c68c:::obj582:head local-lis/les=112/113 n=223 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6131e0b1:::obj290:head) 2026-02-11T23:31:17.548+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.548+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6131e0b1:::obj290:head) v3 finish latency 0.000010 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.548+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.549+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.549+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.549+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5db00 con 0x3e4a000 2026-02-11T23:31:17.549+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:614752af:::obj331:head)]) -- 0x4e0d800 con 0x3e4a000 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.549+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 496 ==== MOSDPGPush(4.0 114/112 [PushOp(4:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183040 con 0x3e4a000 2026-02-11T23:31:17.549+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.549+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.549+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.549+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 497 ==== pg_backfill(progress 4.0 e 114/114 lb 4:614752af:::obj331:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.549+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.549+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:614752af:::obj331:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 114 2026-02-11T23:31:17.549+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:614752af:::obj331:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.549+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.549+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.549+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.549+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.549+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.549+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:614752af:::obj331:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.549+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:61741462:::obj327:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.549+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:61741462:::obj327:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.549+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:61741462:::obj327:head 2026-02-11T23:31:17.549+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:61741462:::obj327:head v 107'327 2026-02-11T23:31:17.549+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.549+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.549+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:61741462:::obj327:head, version: 107'327, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61741462:::obj327:head@107'327, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.549+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:614752af:::obj331:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.549+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:614752af:::obj331:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:614752af:::obj331:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.549+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.549+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:614752af:::obj331:head) v3 2026-02-11T23:31:17.549+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6131e0b1:::obj290:head local-lis/les=112/113 n=224 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:614752af:::obj331:head) 2026-02-11T23:31:17.549+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.549+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:614752af:::obj331:head) v3 finish latency 0.000010 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.549+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.549+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0cf00 con 0x3e4a000 2026-02-11T23:31:17.549+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.549+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:61741462:::obj327:head)]) -- 0x4e0dc80 con 0x3e4a000 2026-02-11T23:31:17.549+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.550+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 498 ==== MOSDPGPush(4.0 114/112 [PushOp(4:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4680 con 0x3e4a000 2026-02-11T23:31:17.550+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.550+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.550+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 499 ==== pg_backfill(progress 4.0 e 114/114 lb 4:61741462:::obj327:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.550+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:61741462:::obj327:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.550+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:61741462:::obj327:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:61741462:::obj327:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:61dc80df:::obj452:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:61dc80df:::obj452:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:61dc80df:::obj452:head 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:61dc80df:::obj452:head v 107'452 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.550+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:61dc80df:::obj452:head, version: 107'452, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:61dc80df:::obj452:head@107'452, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:61741462:::obj327:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:61741462:::obj327:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:61741462:::obj327:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:61741462:::obj327:head) v3 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:614752af:::obj331:head local-lis/les=112/113 n=225 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:61741462:::obj327:head) 2026-02-11T23:31:17.550+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:61741462:::obj327:head) v3 finish latency 0.000012 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.550+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8e900 con 0x3e4a000 2026-02-11T23:31:17.550+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:61dc80df:::obj452:head)]) -- 0x4e0de00 con 0x3e4a000 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.550+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 500 ==== MOSDPGPush(4.0 114/112 [PushOp(4:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516da00 con 0x3e4a000 2026-02-11T23:31:17.550+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.550+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.550+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 501 ==== pg_backfill(progress 4.0 e 114/114 lb 4:61dc80df:::obj452:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.550+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:61dc80df:::obj452:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.550+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:61dc80df:::obj452:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:61dc80df:::obj452:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:622f1330:::obj230:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:622f1330:::obj230:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:622f1330:::obj230:head 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:622f1330:::obj230:head v 107'230 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.550+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.550+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:622f1330:::obj230:head, version: 107'230, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:622f1330:::obj230:head@107'230, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:61dc80df:::obj452:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:61dc80df:::obj452:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 114/114 lb 4:61dc80df:::obj452:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:61dc80df:::obj452:head) v3 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61741462:::obj327:head local-lis/les=112/113 n=226 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:61dc80df:::obj452:head) 2026-02-11T23:31:17.550+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:61dc80df:::obj452:head) v3 finish latency 0.000012 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.550+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.550+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.550+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e0cd80 con 0x3e4a000 2026-02-11T23:31:17.550+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:622f1330:::obj230:head)]) -- 0x4e0c480 con 0x3e4a000 2026-02-11T23:31:17.550+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.551+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 502 ==== MOSDPGPush(4.0 114/112 [PushOp(4:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182000 con 0x3e4a000 2026-02-11T23:31:17.551+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.551+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.551+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 503 ==== pg_backfill(progress 4.0 e 114/114 lb 4:622f1330:::obj230:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.551+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:622f1330:::obj230:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.551+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:622f1330:::obj230:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:622f1330:::obj230:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:624e19c5:::obj464:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:624e19c5:::obj464:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:624e19c5:::obj464:head 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:624e19c5:::obj464:head v 107'464 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.551+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:624e19c5:::obj464:head, version: 107'464, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:624e19c5:::obj464:head@107'464, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:622f1330:::obj230:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.551+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:622f1330:::obj230:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 114/114 lb 4:622f1330:::obj230:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.551+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.551+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:622f1330:::obj230:head) v3 2026-02-11T23:31:17.551+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:61dc80df:::obj452:head local-lis/les=112/113 n=227 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:622f1330:::obj230:head) 2026-02-11T23:31:17.551+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.551+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:622f1330:::obj230:head) v3 finish latency 0.000010 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.551+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.551+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d980 con 0x3e4a000 2026-02-11T23:31:17.551+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:624e19c5:::obj464:head)]) -- 0x4e0c600 con 0x3e4a000 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.551+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 504 ==== MOSDPGPush(4.0 114/112 [PushOp(4:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51824e0 con 0x3e4a000 2026-02-11T23:31:17.551+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.551+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.551+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 505 ==== pg_backfill(progress 4.0 e 114/114 lb 4:624e19c5:::obj464:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.551+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:624e19c5:::obj464:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.551+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:624e19c5:::obj464:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:624e19c5:::obj464:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:62eb9ce0:::obj205:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:62eb9ce0:::obj205:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:62eb9ce0:::obj205:head 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:62eb9ce0:::obj205:head v 107'205 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.551+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.551+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:62eb9ce0:::obj205:head, version: 107'205, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:62eb9ce0:::obj205:head@107'205, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.551+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:624e19c5:::obj464:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.551+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:624e19c5:::obj464:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 114/114 lb 4:624e19c5:::obj464:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.551+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.551+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:624e19c5:::obj464:head) v3 2026-02-11T23:31:17.551+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:622f1330:::obj230:head local-lis/les=112/113 n=228 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:624e19c5:::obj464:head) 2026-02-11T23:31:17.551+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.551+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:624e19c5:::obj464:head) v3 finish latency 0.000011 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.551+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.552+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.552+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.552+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.552+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:62eb9ce0:::obj205:head)]) -- 0x4b7c180 con 0x3e4a000 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.552+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 506 ==== MOSDPGPush(4.0 114/112 [PushOp(4:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3e4a000 2026-02-11T23:31:17.552+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.552+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.552+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.552+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 507 ==== pg_backfill(progress 4.0 e 114/114 lb 4:62eb9ce0:::obj205:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163b80 con 0x3e4a000 2026-02-11T23:31:17.552+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.552+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:62eb9ce0:::obj205:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.552+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.552+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:62eb9ce0:::obj205:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.552+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.552+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.552+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.552+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:62eb9ce0:::obj205:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.552+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.552+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:63191178:::obj308:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.552+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:63191178:::obj308:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.552+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:63191178:::obj308:head 2026-02-11T23:31:17.552+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:63191178:::obj308:head v 107'308 2026-02-11T23:31:17.552+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.552+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.552+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:63191178:::obj308:head, version: 107'308, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:63191178:::obj308:head@107'308, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.552+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:62eb9ce0:::obj205:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.552+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:62eb9ce0:::obj205:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 114/114 lb 4:62eb9ce0:::obj205:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.552+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.552+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:62eb9ce0:::obj205:head) v3 2026-02-11T23:31:17.552+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:624e19c5:::obj464:head local-lis/les=112/113 n=229 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:62eb9ce0:::obj205:head) 2026-02-11T23:31:17.552+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.552+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:62eb9ce0:::obj205:head) v3 finish latency 0.000010 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.552+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.552+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.552+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.552+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:63191178:::obj308:head)]) -- 0x4c80600 con 0x3e4a000 2026-02-11T23:31:17.552+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.553+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 508 ==== MOSDPGPush(4.0 114/112 [PushOp(4:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2680 con 0x3e4a000 2026-02-11T23:31:17.553+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.553+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.553+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 509 ==== pg_backfill(progress 4.0 e 114/114 lb 4:63191178:::obj308:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163600 con 0x3e4a000 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.553+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:63191178:::obj308:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.553+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:63191178:::obj308:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:63191178:::obj308:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:640d3668:::obj286:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:640d3668:::obj286:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:640d3668:::obj286:head 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:640d3668:::obj286:head v 107'286 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.553+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:640d3668:::obj286:head, version: 107'286, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:640d3668:::obj286:head@107'286, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:63191178:::obj308:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:63191178:::obj308:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:63191178:::obj308:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:63191178:::obj308:head) v3 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:62eb9ce0:::obj205:head local-lis/les=112/113 n=230 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:63191178:::obj308:head) 2026-02-11T23:31:17.553+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:63191178:::obj308:head) v3 finish latency 0.000010 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.553+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d080 con 0x3e4a000 2026-02-11T23:31:17.553+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:640d3668:::obj286:head)]) -- 0x511a180 con 0x3e4a000 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.553+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 510 ==== MOSDPGPush(4.0 114/112 [PushOp(4:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.553+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.553+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.553+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 511 ==== pg_backfill(progress 4.0 e 114/114 lb 4:640d3668:::obj286:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163080 con 0x3e4a000 2026-02-11T23:31:17.553+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:640d3668:::obj286:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.553+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:640d3668:::obj286:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:640d3668:::obj286:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:649aec78:::obj275:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:649aec78:::obj275:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:649aec78:::obj275:head 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:649aec78:::obj275:head v 107'275 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.553+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.553+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:649aec78:::obj275:head, version: 107'275, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:649aec78:::obj275:head@107'275, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:640d3668:::obj286:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:640d3668:::obj286:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:640d3668:::obj286:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:640d3668:::obj286:head) v3 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:63191178:::obj308:head local-lis/les=112/113 n=231 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:640d3668:::obj286:head) 2026-02-11T23:31:17.553+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:640d3668:::obj286:head) v3 finish latency 0.000009 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.553+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.553+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.553+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d500 con 0x3e4a000 2026-02-11T23:31:17.553+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:649aec78:::obj275:head)]) -- 0x511a480 con 0x3e4a000 2026-02-11T23:31:17.553+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.554+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 512 ==== MOSDPGPush(4.0 114/112 [PushOp(4:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.554+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.554+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.554+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.554+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 513 ==== pg_backfill(progress 4.0 e 114/114 lb 4:649aec78:::obj275:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3e4a000 2026-02-11T23:31:17.554+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.554+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:649aec78:::obj275:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 114 2026-02-11T23:31:17.554+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.554+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.554+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:649aec78:::obj275:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.554+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.554+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:649aec78:::obj275:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.554+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.554+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.554+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.554+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.554+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:64b337fe:::obj71:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.554+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:64b337fe:::obj71:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.554+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:64b337fe:::obj71:head 2026-02-11T23:31:17.554+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:64b337fe:::obj71:head v 107'71 2026-02-11T23:31:17.554+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.554+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.554+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:64b337fe:::obj71:head, version: 107'71, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64b337fe:::obj71:head@107'71, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.554+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.554+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.554+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:649aec78:::obj275:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.554+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:649aec78:::obj275:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:649aec78:::obj275:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.554+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.554+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:649aec78:::obj275:head) v3 2026-02-11T23:31:17.554+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:640d3668:::obj286:head local-lis/les=112/113 n=232 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:649aec78:::obj275:head) 2026-02-11T23:31:17.554+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.554+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:649aec78:::obj275:head) v3 finish latency 0.000011 2026-02-11T23:31:17.554+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.554+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.554+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.554+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.554+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.554+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.554+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.554+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8f500 con 0x3e4a000 2026-02-11T23:31:17.554+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:64b337fe:::obj71:head)]) -- 0x511a780 con 0x3e4a000 2026-02-11T23:31:17.554+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.554+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 514 ==== MOSDPGPush(4.0 114/112 [PushOp(4:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.554+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.554+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.554+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.555+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 515 ==== pg_backfill(progress 4.0 e 114/114 lb 4:64b337fe:::obj71:head) ==== 1061+0+0 (unknown 0 0 0) 0x5162580 con 0x3e4a000 2026-02-11T23:31:17.555+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:64b337fe:::obj71:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 114 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.555+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:64b337fe:::obj71:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:64b337fe:::obj71:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 114/112 [PushOp(4:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:64bc0383:::obj214:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:64bc0383:::obj214:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:64bc0383:::obj214:head 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:64bc0383:::obj214:head v 107'214 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.555+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:64bc0383:::obj214:head, version: 107'214, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64bc0383:::obj214:head@107'214, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:64b337fe:::obj71:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:64b337fe:::obj71:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 114/114 lb 4:64b337fe:::obj71:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:64b337fe:::obj71:head) v3 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:649aec78:::obj275:head local-lis/les=112/113 n=233 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:64b337fe:::obj71:head) 2026-02-11T23:31:17.555+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:64b337fe:::obj71:head) v3 finish latency 0.000009 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.555+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4fcf200 con 0x3e4a000 2026-02-11T23:31:17.555+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:64bc0383:::obj214:head)]) -- 0x511a900 con 0x3e4a000 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.555+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 516 ==== MOSDPGPush(4.0 114/112 [PushOp(4:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.555+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.555+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.555+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 517 ==== pg_backfill(progress 4.0 e 114/114 lb 4:64bc0383:::obj214:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162000 con 0x3e4a000 2026-02-11T23:31:17.555+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:64bc0383:::obj214:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.555+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:64bc0383:::obj214:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:64bc0383:::obj214:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:64c1dddc:::obj83:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:64c1dddc:::obj83:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:64c1dddc:::obj83:head 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:64c1dddc:::obj83:head v 107'83 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.555+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.555+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:64c1dddc:::obj83:head, version: 107'83, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:64c1dddc:::obj83:head@107'83, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:64bc0383:::obj214:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:64bc0383:::obj214:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:64bc0383:::obj214:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:64bc0383:::obj214:head) v3 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64b337fe:::obj71:head local-lis/les=112/113 n=234 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:64bc0383:::obj214:head) 2026-02-11T23:31:17.555+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:64bc0383:::obj214:head) v3 finish latency 0.000009 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.555+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.555+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.555+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1fc80 con 0x3e4a000 2026-02-11T23:31:17.555+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:64c1dddc:::obj83:head)]) -- 0x511ac00 con 0x3e4a000 2026-02-11T23:31:17.555+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.556+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 518 ==== MOSDPGPush(4.0 114/112 [PushOp(4:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2340 con 0x3e4a000 2026-02-11T23:31:17.556+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.556+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.556+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 519 ==== pg_backfill(progress 4.0 e 114/114 lb 4:64c1dddc:::obj83:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3e4a000 2026-02-11T23:31:17.556+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:64c1dddc:::obj83:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.556+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:64c1dddc:::obj83:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:64c1dddc:::obj83:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:657806de:::obj544:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:657806de:::obj544:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:657806de:::obj544:head 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:657806de:::obj544:head v 107'544 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.556+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:657806de:::obj544:head, version: 107'544, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:657806de:::obj544:head@107'544, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:64c1dddc:::obj83:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.556+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:64c1dddc:::obj83:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:64c1dddc:::obj83:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.556+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.556+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:64c1dddc:::obj83:head) v3 2026-02-11T23:31:17.556+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64bc0383:::obj214:head local-lis/les=112/113 n=235 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:64c1dddc:::obj83:head) 2026-02-11T23:31:17.556+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.556+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:64c1dddc:::obj83:head) v3 finish latency 0.000010 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.556+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.556+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d680 con 0x3e4a000 2026-02-11T23:31:17.556+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:657806de:::obj544:head)]) -- 0x511ad80 con 0x3e4a000 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.556+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 520 ==== MOSDPGPush(4.0 114/112 [PushOp(4:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3e4a000 2026-02-11T23:31:17.556+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.556+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.556+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 521 ==== pg_backfill(progress 4.0 e 114/114 lb 4:657806de:::obj544:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3e4a000 2026-02-11T23:31:17.556+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:657806de:::obj544:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.556+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:657806de:::obj544:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.556+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:657806de:::obj544:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:65a1355c:::obj536:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.556+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:65a1355c:::obj536:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:65a1355c:::obj536:head 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:65a1355c:::obj536:head v 107'536 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.556+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.556+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:65a1355c:::obj536:head, version: 107'536, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:65a1355c:::obj536:head@107'536, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.557+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:657806de:::obj544:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:657806de:::obj544:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:657806de:::obj544:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:657806de:::obj544:head) v3 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:64c1dddc:::obj83:head local-lis/les=112/113 n=236 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:657806de:::obj544:head) 2026-02-11T23:31:17.557+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:657806de:::obj544:head) v3 finish latency 0.000010 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.557+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.557+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1f980 con 0x3e4a000 2026-02-11T23:31:17.557+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:65a1355c:::obj536:head)]) -- 0x511b080 con 0x3e4a000 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.557+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 522 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3e4a000 2026-02-11T23:31:17.557+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.557+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.557+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.557+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 523 ==== pg_backfill(progress 4.0 e 114/114 lb 4:65a1355c:::obj536:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3e4a000 2026-02-11T23:31:17.557+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.557+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:65a1355c:::obj536:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.557+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:65a1355c:::obj536:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.557+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.557+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.557+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.557+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:65a1355c:::obj536:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.557+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.557+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.557+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6623e3eb:::obj517:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.557+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6623e3eb:::obj517:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.557+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6623e3eb:::obj517:head 2026-02-11T23:31:17.557+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6623e3eb:::obj517:head v 107'517 2026-02-11T23:31:17.557+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.557+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.557+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6623e3eb:::obj517:head, version: 107'517, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6623e3eb:::obj517:head@107'517, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.557+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:65a1355c:::obj536:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:65a1355c:::obj536:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 114/114 lb 4:65a1355c:::obj536:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:65a1355c:::obj536:head) v3 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:657806de:::obj544:head local-lis/les=112/113 n=237 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:65a1355c:::obj536:head) 2026-02-11T23:31:17.557+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:65a1355c:::obj536:head) v3 finish latency 0.000010 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.557+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.557+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.557+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8ec00 con 0x3e4a000 2026-02-11T23:31:17.557+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6623e3eb:::obj517:head)]) -- 0x511b380 con 0x3e4a000 2026-02-11T23:31:17.557+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.558+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 524 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3e4a000 2026-02-11T23:31:17.558+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.558+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.558+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 525 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6623e3eb:::obj517:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3e4a000 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.558+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6623e3eb:::obj517:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.558+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6623e3eb:::obj517:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6623e3eb:::obj517:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6681cfe7:::obj102:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6681cfe7:::obj102:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6681cfe7:::obj102:head 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6681cfe7:::obj102:head v 107'102 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.558+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6681cfe7:::obj102:head, version: 107'102, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6681cfe7:::obj102:head@107'102, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6623e3eb:::obj517:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6623e3eb:::obj517:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 114/114 lb 4:6623e3eb:::obj517:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6623e3eb:::obj517:head) v3 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:65a1355c:::obj536:head local-lis/les=112/113 n=238 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6623e3eb:::obj517:head) 2026-02-11T23:31:17.558+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6623e3eb:::obj517:head) v3 finish latency 0.000010 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.558+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1f680 con 0x3e4a000 2026-02-11T23:31:17.558+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6681cfe7:::obj102:head)]) -- 0x511b680 con 0x3e4a000 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.558+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 526 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3e4a000 2026-02-11T23:31:17.558+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.558+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.558+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 527 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6681cfe7:::obj102:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d700 con 0x3e4a000 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.558+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6681cfe7:::obj102:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.558+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6681cfe7:::obj102:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6681cfe7:::obj102:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6727c4f6:::obj39:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6727c4f6:::obj39:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6727c4f6:::obj39:head 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6727c4f6:::obj39:head v 107'39 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.558+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.558+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6727c4f6:::obj39:head, version: 107'39, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6727c4f6:::obj39:head@107'39, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6681cfe7:::obj102:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6681cfe7:::obj102:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:6681cfe7:::obj102:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6681cfe7:::obj102:head) v3 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6623e3eb:::obj517:head local-lis/les=112/113 n=239 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6681cfe7:::obj102:head) 2026-02-11T23:31:17.558+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6681cfe7:::obj102:head) v3 finish latency 0.000010 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.558+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.558+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.558+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d380 con 0x3e4a000 2026-02-11T23:31:17.558+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6727c4f6:::obj39:head)]) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.558+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.559+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 528 ==== MOSDPGPush(4.0 114/112 [PushOp(4:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43ebba0 con 0x3e4a000 2026-02-11T23:31:17.559+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.559+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.559+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 529 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6727c4f6:::obj39:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3e4a000 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.559+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6727c4f6:::obj39:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.559+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6727c4f6:::obj39:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6727c4f6:::obj39:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:67a26c0a:::obj53:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:67a26c0a:::obj53:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:67a26c0a:::obj53:head 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:67a26c0a:::obj53:head v 107'53 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.559+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:67a26c0a:::obj53:head, version: 107'53, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:67a26c0a:::obj53:head@107'53, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6727c4f6:::obj39:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.559+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6727c4f6:::obj39:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:6727c4f6:::obj39:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.559+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.559+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6727c4f6:::obj39:head) v3 2026-02-11T23:31:17.559+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6681cfe7:::obj102:head local-lis/les=112/113 n=240 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6727c4f6:::obj39:head) 2026-02-11T23:31:17.559+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.559+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6727c4f6:::obj39:head) v3 finish latency 0.000011 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.559+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.559+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1f380 con 0x3e4a000 2026-02-11T23:31:17.559+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:67a26c0a:::obj53:head)]) -- 0x511bb00 con 0x3e4a000 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.559+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 530 ==== MOSDPGPush(4.0 114/112 [PushOp(4:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.559+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.559+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.559+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 531 ==== pg_backfill(progress 4.0 e 114/114 lb 4:67a26c0a:::obj53:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3e4a000 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.559+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:67a26c0a:::obj53:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.559+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:67a26c0a:::obj53:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:67a26c0a:::obj53:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:685e1da9:::obj264:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:685e1da9:::obj264:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:685e1da9:::obj264:head 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:685e1da9:::obj264:head v 107'264 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.559+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.559+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:685e1da9:::obj264:head, version: 107'264, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e1da9:::obj264:head@107'264, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.559+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:67a26c0a:::obj53:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.559+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:67a26c0a:::obj53:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 114/114 lb 4:67a26c0a:::obj53:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.559+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.559+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:67a26c0a:::obj53:head) v3 2026-02-11T23:31:17.559+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6727c4f6:::obj39:head local-lis/les=112/113 n=241 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:67a26c0a:::obj53:head) 2026-02-11T23:31:17.559+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.559+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:67a26c0a:::obj53:head) v3 finish latency 0.000011 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.559+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.560+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.560+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.560+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x49d7800 con 0x3e4a000 2026-02-11T23:31:17.560+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:685e1da9:::obj264:head)]) -- 0x511bc80 con 0x3e4a000 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.560+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 532 ==== MOSDPGPush(4.0 114/112 [PushOp(4:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.560+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.560+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.560+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.560+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 533 ==== pg_backfill(progress 4.0 e 114/114 lb 4:685e1da9:::obj264:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3e4a000 2026-02-11T23:31:17.560+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.560+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:685e1da9:::obj264:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.560+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:685e1da9:::obj264:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.560+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.560+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.560+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.560+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.560+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:685e1da9:::obj264:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.560+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.560+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:685e6e29:::obj176:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.560+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:685e6e29:::obj176:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.560+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:685e6e29:::obj176:head 2026-02-11T23:31:17.560+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:685e6e29:::obj176:head v 107'176 2026-02-11T23:31:17.560+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.560+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.560+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:685e6e29:::obj176:head, version: 107'176, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:685e6e29:::obj176:head@107'176, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.560+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:685e1da9:::obj264:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.560+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:685e1da9:::obj264:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 114/114 lb 4:685e1da9:::obj264:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.560+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.560+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:685e1da9:::obj264:head) v3 2026-02-11T23:31:17.560+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:67a26c0a:::obj53:head local-lis/les=112/113 n=242 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:685e1da9:::obj264:head) 2026-02-11T23:31:17.560+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.560+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:685e1da9:::obj264:head) v3 finish latency 0.000011 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.560+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.560+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.560+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1f080 con 0x3e4a000 2026-02-11T23:31:17.560+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:685e6e29:::obj176:head)]) -- 0x4c1fc80 con 0x3e4a000 2026-02-11T23:31:17.560+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.560+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 534 ==== MOSDPGPush(4.0 114/112 [PushOp(4:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.561+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 114 2026-02-11T23:31:17.561+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.561+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 535 ==== pg_backfill(progress 4.0 e 114/114 lb 4:685e6e29:::obj176:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3e4a000 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.561+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:685e6e29:::obj176:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.561+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:685e6e29:::obj176:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:685e6e29:::obj176:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:68743261:::obj370:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:68743261:::obj370:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:68743261:::obj370:head 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:68743261:::obj370:head v 107'370 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.561+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:68743261:::obj370:head, version: 107'370, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68743261:::obj370:head@107'370, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:685e6e29:::obj176:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:685e6e29:::obj176:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 114/114 lb 4:685e6e29:::obj176:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:685e6e29:::obj176:head) v3 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e1da9:::obj264:head local-lis/les=112/113 n=243 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:685e6e29:::obj176:head) 2026-02-11T23:31:17.561+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:685e6e29:::obj176:head) v3 finish latency 0.000011 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.561+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.561+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:68743261:::obj370:head)]) -- 0x4fcf200 con 0x3e4a000 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.561+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 536 ==== MOSDPGPush(4.0 114/112 [PushOp(4:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.561+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.561+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.561+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 537 ==== pg_backfill(progress 4.0 e 114/114 lb 4:68743261:::obj370:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3e4a000 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.561+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:68743261:::obj370:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.561+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:68743261:::obj370:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:68743261:::obj370:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:68869f42:::obj433:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:68869f42:::obj433:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:68869f42:::obj433:head 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:68869f42:::obj433:head v 107'433 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.561+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.561+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:68869f42:::obj433:head, version: 107'433, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:68869f42:::obj433:head@107'433, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:68743261:::obj370:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:68743261:::obj370:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 114/114 lb 4:68743261:::obj370:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:68743261:::obj370:head) v3 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:685e6e29:::obj176:head local-lis/les=112/113 n=244 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:68743261:::obj370:head) 2026-02-11T23:31:17.561+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:68743261:::obj370:head) v3 finish latency 0.000011 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.561+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.561+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.561+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1ed80 con 0x3e4a000 2026-02-11T23:31:17.561+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:68869f42:::obj433:head)]) -- 0x4d8f500 con 0x3e4a000 2026-02-11T23:31:17.561+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.562+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 538 ==== MOSDPGPush(4.0 114/112 [PushOp(4:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2680 con 0x3e4a000 2026-02-11T23:31:17.562+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.562+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.562+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 539 ==== pg_backfill(progress 4.0 e 114/114 lb 4:68869f42:::obj433:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3e4a000 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.562+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:68869f42:::obj433:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.562+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:68869f42:::obj433:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:68869f42:::obj433:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:69182b79:::obj301:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:69182b79:::obj301:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:69182b79:::obj301:head 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:69182b79:::obj301:head v 107'301 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.562+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:69182b79:::obj301:head, version: 107'301, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69182b79:::obj301:head@107'301, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:68869f42:::obj433:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:68869f42:::obj433:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:68869f42:::obj433:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:68869f42:::obj433:head) v3 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68743261:::obj370:head local-lis/les=112/113 n=245 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:68869f42:::obj433:head) 2026-02-11T23:31:17.562+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:68869f42:::obj433:head) v3 finish latency 0.000010 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.562+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8ef00 con 0x3e4a000 2026-02-11T23:31:17.562+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:69182b79:::obj301:head)]) -- 0x4e5d500 con 0x3e4a000 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.562+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 540 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3e4a000 2026-02-11T23:31:17.562+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.562+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.562+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 541 ==== pg_backfill(progress 4.0 e 114/114 lb 4:69182b79:::obj301:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3e4a000 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.562+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:69182b79:::obj301:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 114 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.562+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:69182b79:::obj301:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:69182b79:::obj301:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6934f76e:::obj318:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6934f76e:::obj318:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6934f76e:::obj318:head 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6934f76e:::obj318:head v 107'318 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.562+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.562+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6934f76e:::obj318:head, version: 107'318, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6934f76e:::obj318:head@107'318, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:69182b79:::obj301:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:69182b79:::obj301:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 114/114 lb 4:69182b79:::obj301:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:69182b79:::obj301:head) v3 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:68869f42:::obj433:head local-lis/les=112/113 n=246 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:69182b79:::obj301:head) 2026-02-11T23:31:17.562+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:69182b79:::obj301:head) v3 finish latency 0.000010 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.562+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.562+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.562+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1ea80 con 0x3e4a000 2026-02-11T23:31:17.562+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6934f76e:::obj318:head)]) -- 0x4e5d080 con 0x3e4a000 2026-02-11T23:31:17.562+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.563+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 542 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51824e0 con 0x3e4a000 2026-02-11T23:31:17.563+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.563+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.563+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 543 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6934f76e:::obj318:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3e4a000 2026-02-11T23:31:17.563+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6934f76e:::obj318:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.563+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6934f76e:::obj318:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.563+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.563+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.563+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.563+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6934f76e:::obj318:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.563+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.563+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.563+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6971da6a:::obj293:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6971da6a:::obj293:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.563+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6971da6a:::obj293:head 2026-02-11T23:31:17.563+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6971da6a:::obj293:head v 107'293 2026-02-11T23:31:17.563+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.563+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.563+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6971da6a:::obj293:head, version: 107'293, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6971da6a:::obj293:head@107'293, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.563+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.563+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6934f76e:::obj318:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.563+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6934f76e:::obj318:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 114/114 lb 4:6934f76e:::obj318:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.563+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.563+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6934f76e:::obj318:head) v3 2026-02-11T23:31:17.563+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69182b79:::obj301:head local-lis/les=112/113 n=247 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6934f76e:::obj318:head) 2026-02-11T23:31:17.563+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.563+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6934f76e:::obj318:head) v3 finish latency 0.000020 2026-02-11T23:31:17.563+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.563+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.563+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.563+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8f200 con 0x3e4a000 2026-02-11T23:31:17.563+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6971da6a:::obj293:head)]) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.563+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 544 ==== MOSDPGPush(4.0 114/112 [PushOp(4:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182000 con 0x3e4a000 2026-02-11T23:31:17.563+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.563+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.563+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.563+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 545 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6971da6a:::obj293:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3e4a000 2026-02-11T23:31:17.563+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.563+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6971da6a:::obj293:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.563+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.563+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6971da6a:::obj293:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.563+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.563+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.563+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6971da6a:::obj293:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:698e6c1e:::obj207:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.564+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:698e6c1e:::obj207:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:698e6c1e:::obj207:head 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:698e6c1e:::obj207:head v 107'207 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.564+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:698e6c1e:::obj207:head, version: 107'207, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:698e6c1e:::obj207:head@107'207, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.564+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6971da6a:::obj293:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6971da6a:::obj293:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:6971da6a:::obj293:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6971da6a:::obj293:head) v3 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6934f76e:::obj318:head local-lis/les=112/113 n=248 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6971da6a:::obj293:head) 2026-02-11T23:31:17.564+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6971da6a:::obj293:head) v3 finish latency 0.000012 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.564+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.564+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.564+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:698e6c1e:::obj207:head)]) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.564+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 546 ==== MOSDPGPush(4.0 114/112 [PushOp(4:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516da00 con 0x3e4a000 2026-02-11T23:31:17.564+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.564+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.564+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.564+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 547 ==== pg_backfill(progress 4.0 e 114/114 lb 4:698e6c1e:::obj207:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3e4a000 2026-02-11T23:31:17.564+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.564+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:698e6c1e:::obj207:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 114 2026-02-11T23:31:17.564+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:698e6c1e:::obj207:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.564+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.564+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 114/112 [PushOp(4:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:698e6c1e:::obj207:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:699d4764:::obj50:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.564+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:699d4764:::obj50:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:699d4764:::obj50:head 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:699d4764:::obj50:head v 107'50 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.564+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.564+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:699d4764:::obj50:head, version: 107'50, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:699d4764:::obj50:head@107'50, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.564+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:698e6c1e:::obj207:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:698e6c1e:::obj207:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 114/114 lb 4:698e6c1e:::obj207:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:698e6c1e:::obj207:head) v3 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6971da6a:::obj293:head local-lis/les=112/113 n=249 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:698e6c1e:::obj207:head) 2026-02-11T23:31:17.564+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:698e6c1e:::obj207:head) v3 finish latency 0.000012 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.564+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.564+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.564+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x49b8c00 con 0x3e4a000 2026-02-11T23:31:17.564+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:699d4764:::obj50:head)]) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.564+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.565+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 548 ==== MOSDPGPush(4.0 114/112 [PushOp(4:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4680 con 0x3e4a000 2026-02-11T23:31:17.565+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.565+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.565+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 549 ==== pg_backfill(progress 4.0 e 114/114 lb 4:699d4764:::obj50:head) ==== 1061+0+0 (unknown 0 0 0) 0x48bc100 con 0x3e4a000 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.565+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:699d4764:::obj50:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.565+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:699d4764:::obj50:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:699d4764:::obj50:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:69f21cc4:::obj383:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:69f21cc4:::obj383:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:69f21cc4:::obj383:head 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:69f21cc4:::obj383:head v 107'383 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.565+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:69f21cc4:::obj383:head, version: 107'383, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:69f21cc4:::obj383:head@107'383, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:699d4764:::obj50:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.565+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:699d4764:::obj50:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 114/114 lb 4:699d4764:::obj50:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.565+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.565+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:699d4764:::obj50:head) v3 2026-02-11T23:31:17.565+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:698e6c1e:::obj207:head local-lis/les=112/113 n=250 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:699d4764:::obj50:head) 2026-02-11T23:31:17.565+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.565+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:699d4764:::obj50:head) v3 finish latency 0.000010 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.565+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.565+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1e300 con 0x3e4a000 2026-02-11T23:31:17.565+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:69f21cc4:::obj383:head)]) -- 0x4e0c600 con 0x3e4a000 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.565+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 550 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183040 con 0x3e4a000 2026-02-11T23:31:17.565+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.565+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.565+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 551 ==== pg_backfill(progress 4.0 e 114/114 lb 4:69f21cc4:::obj383:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3e4a000 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.565+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:69f21cc4:::obj383:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.565+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:69f21cc4:::obj383:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:69f21cc4:::obj383:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6c7994f7:::obj432:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6c7994f7:::obj432:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6c7994f7:::obj432:head 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6c7994f7:::obj432:head v 107'432 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.565+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.565+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6c7994f7:::obj432:head, version: 107'432, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6c7994f7:::obj432:head@107'432, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.565+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:69f21cc4:::obj383:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.565+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:69f21cc4:::obj383:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 114/114 lb 4:69f21cc4:::obj383:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.565+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.565+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:69f21cc4:::obj383:head) v3 2026-02-11T23:31:17.565+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:699d4764:::obj50:head local-lis/les=112/113 n=251 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:69f21cc4:::obj383:head) 2026-02-11T23:31:17.565+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.565+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:69f21cc4:::obj383:head) v3 finish latency 0.000011 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.565+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.566+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1e300 con 0x3e4a000 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.566+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6c7994f7:::obj432:head)]) -- 0x4e0c480 con 0x3e4a000 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.566+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 552 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.566+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 114 2026-02-11T23:31:17.566+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.566+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 553 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6c7994f7:::obj432:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.566+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6c7994f7:::obj432:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.566+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6c7994f7:::obj432:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6c7994f7:::obj432:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6dc7b1fd:::obj49:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6dc7b1fd:::obj49:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6dc7b1fd:::obj49:head 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6dc7b1fd:::obj49:head v 107'49 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.566+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6dc7b1fd:::obj49:head, version: 107'49, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6dc7b1fd:::obj49:head@107'49, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6c7994f7:::obj432:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6c7994f7:::obj432:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 114/114 lb 4:6c7994f7:::obj432:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6c7994f7:::obj432:head) v3 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:69f21cc4:::obj383:head local-lis/les=112/113 n=252 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6c7994f7:::obj432:head) 2026-02-11T23:31:17.566+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6c7994f7:::obj432:head) v3 finish latency 0.000010 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.566+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.566+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6dc7b1fd:::obj49:head)]) -- 0x4e0de00 con 0x3e4a000 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.566+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 554 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.566+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.566+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.566+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 555 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6dc7b1fd:::obj49:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164100 con 0x3e4a000 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.566+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6dc7b1fd:::obj49:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.566+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6dc7b1fd:::obj49:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6dc7b1fd:::obj49:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6e0bfbf2:::obj372:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6e0bfbf2:::obj372:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6e0bfbf2:::obj372:head 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6e0bfbf2:::obj372:head v 107'372 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.566+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.566+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6e0bfbf2:::obj372:head, version: 107'372, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6e0bfbf2:::obj372:head@107'372, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.566+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6dc7b1fd:::obj49:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6dc7b1fd:::obj49:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 114/114 lb 4:6dc7b1fd:::obj49:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6dc7b1fd:::obj49:head) v3 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6c7994f7:::obj432:head local-lis/les=112/113 n=253 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6dc7b1fd:::obj49:head) 2026-02-11T23:31:17.566+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.566+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6dc7b1fd:::obj49:head) v3 finish latency 0.000012 2026-02-11T23:31:17.566+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.567+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.567+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.567+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x49b8c00 con 0x3e4a000 2026-02-11T23:31:17.567+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6e0bfbf2:::obj372:head)]) -- 0x4d8e900 con 0x3e4a000 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.567+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 556 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.567+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.567+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.567+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.567+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 557 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6e0bfbf2:::obj372:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.567+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.567+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6e0bfbf2:::obj372:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.567+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6e0bfbf2:::obj372:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.567+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.567+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.567+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.567+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6e0bfbf2:::obj372:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.567+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.567+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.567+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6ec34796:::obj408:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.567+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6ec34796:::obj408:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.567+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6ec34796:::obj408:head 2026-02-11T23:31:17.567+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6ec34796:::obj408:head v 107'408 2026-02-11T23:31:17.567+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.567+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.567+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6ec34796:::obj408:head, version: 107'408, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec34796:::obj408:head@107'408, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.567+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6e0bfbf2:::obj372:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.567+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6e0bfbf2:::obj372:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 114/114 lb 4:6e0bfbf2:::obj372:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.567+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.567+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6e0bfbf2:::obj372:head) v3 2026-02-11T23:31:17.567+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6dc7b1fd:::obj49:head local-lis/les=112/113 n=254 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6e0bfbf2:::obj372:head) 2026-02-11T23:31:17.567+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.567+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6e0bfbf2:::obj372:head) v3 finish latency 0.000010 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.567+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.567+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.567+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.567+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6ec34796:::obj408:head)]) -- 0x4e0cf00 con 0x3e4a000 2026-02-11T23:31:17.567+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.568+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 558 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.568+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.568+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.568+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 559 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6ec34796:::obj408:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.568+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6ec34796:::obj408:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.568+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6ec34796:::obj408:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6ec34796:::obj408:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6ec93c40:::obj366:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6ec93c40:::obj366:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6ec93c40:::obj366:head 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6ec93c40:::obj366:head v 107'366 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.568+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6ec93c40:::obj366:head, version: 107'366, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec93c40:::obj366:head@107'366, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6ec34796:::obj408:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6ec34796:::obj408:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 114/114 lb 4:6ec34796:::obj408:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6ec34796:::obj408:head) v3 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6e0bfbf2:::obj372:head local-lis/les=112/113 n=255 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6ec34796:::obj408:head) 2026-02-11T23:31:17.568+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6ec34796:::obj408:head) v3 finish latency 0.000011 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.568+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.568+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6ec93c40:::obj366:head)]) -- 0x4e5db00 con 0x3e4a000 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.568+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 560 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3e4a000 2026-02-11T23:31:17.568+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.568+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.568+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 561 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6ec93c40:::obj366:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.568+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6ec93c40:::obj366:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 114 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.568+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6ec93c40:::obj366:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 114/112 [PushOp(4:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6ec93c40:::obj366:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6ec9e2eb:::obj591:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6ec9e2eb:::obj591:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6ec9e2eb:::obj591:head 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6ec9e2eb:::obj591:head v 107'591 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.568+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.568+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6ec9e2eb:::obj591:head, version: 107'591, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6ec9e2eb:::obj591:head@107'591, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6ec93c40:::obj366:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6ec93c40:::obj366:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:6ec93c40:::obj366:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6ec93c40:::obj366:head) v3 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec34796:::obj408:head local-lis/les=112/113 n=256 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6ec93c40:::obj366:head) 2026-02-11T23:31:17.568+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6ec93c40:::obj366:head) v3 finish latency 0.000011 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.568+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.568+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.568+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.568+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6ec9e2eb:::obj591:head)]) -- 0x4e0d080 con 0x3e4a000 2026-02-11T23:31:17.568+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.569+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 562 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3e4a000 2026-02-11T23:31:17.569+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.569+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.569+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 563 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6ec9e2eb:::obj591:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.569+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6ec9e2eb:::obj591:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.569+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6ec9e2eb:::obj591:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6ec9e2eb:::obj591:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6f4176ae:::obj571:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6f4176ae:::obj571:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6f4176ae:::obj571:head 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6f4176ae:::obj571:head v 107'571 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.569+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6f4176ae:::obj571:head, version: 107'571, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f4176ae:::obj571:head@107'571, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6ec9e2eb:::obj591:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.569+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6ec9e2eb:::obj591:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 114/114 lb 4:6ec9e2eb:::obj591:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.569+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.569+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6ec9e2eb:::obj591:head) v3 2026-02-11T23:31:17.569+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec93c40:::obj366:head local-lis/les=112/113 n=257 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6ec9e2eb:::obj591:head) 2026-02-11T23:31:17.569+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.569+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6ec9e2eb:::obj591:head) v3 finish latency 0.000010 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.569+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.569+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8f200 con 0x3e4a000 2026-02-11T23:31:17.569+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6f4176ae:::obj571:head)]) -- 0x4d8e780 con 0x3e4a000 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.569+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 564 ==== MOSDPGPush(4.0 114/112 [PushOp(4:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.569+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.569+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.569+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 565 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6f4176ae:::obj571:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.569+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6f4176ae:::obj571:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.569+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6f4176ae:::obj571:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6f4176ae:::obj571:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:6f5862a4:::obj430:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:6f5862a4:::obj430:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:6f5862a4:::obj430:head 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:6f5862a4:::obj430:head v 107'430 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.569+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.569+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:6f5862a4:::obj430:head, version: 107'430, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:6f5862a4:::obj430:head@107'430, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.569+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6f4176ae:::obj571:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.569+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6f4176ae:::obj571:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 114/114 lb 4:6f4176ae:::obj571:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.569+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.569+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6f4176ae:::obj571:head) v3 2026-02-11T23:31:17.569+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6ec9e2eb:::obj591:head local-lis/les=112/113 n=258 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6f4176ae:::obj571:head) 2026-02-11T23:31:17.569+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.569+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6f4176ae:::obj571:head) v3 finish latency 0.000010 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.569+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.570+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.570+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.570+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d080 con 0x3e4a000 2026-02-11T23:31:17.570+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:6f5862a4:::obj430:head)]) -- 0x4d8e180 con 0x3e4a000 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.570+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 566 ==== MOSDPGPush(4.0 114/112 [PushOp(4:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3e4a000 2026-02-11T23:31:17.570+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.570+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.570+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.570+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 567 ==== pg_backfill(progress 4.0 e 114/114 lb 4:6f5862a4:::obj430:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.570+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.570+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:6f5862a4:::obj430:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.570+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6f5862a4:::obj430:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.570+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.570+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.570+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.570+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6f5862a4:::obj430:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.570+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.570+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.570+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:70360d67:::obj288:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.570+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:70360d67:::obj288:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.570+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:70360d67:::obj288:head 2026-02-11T23:31:17.570+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:70360d67:::obj288:head v 107'288 2026-02-11T23:31:17.570+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.570+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.570+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:70360d67:::obj288:head, version: 107'288, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:70360d67:::obj288:head@107'288, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.570+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:6f5862a4:::obj430:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.570+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6f5862a4:::obj430:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:6f5862a4:::obj430:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.570+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.570+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:6f5862a4:::obj430:head) v3 2026-02-11T23:31:17.570+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f4176ae:::obj571:head local-lis/les=112/113 n=259 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:6f5862a4:::obj430:head) 2026-02-11T23:31:17.570+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.570+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:6f5862a4:::obj430:head) v3 finish latency 0.000011 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.570+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.570+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.570+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1ea80 con 0x3e4a000 2026-02-11T23:31:17.570+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:70360d67:::obj288:head)]) -- 0x4b7d500 con 0x3e4a000 2026-02-11T23:31:17.570+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.571+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 568 ==== MOSDPGPush(4.0 114/112 [PushOp(4:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.571+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 114 2026-02-11T23:31:17.571+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.571+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 569 ==== pg_backfill(progress 4.0 e 114/114 lb 4:70360d67:::obj288:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.571+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:70360d67:::obj288:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.571+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:70360d67:::obj288:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:70360d67:::obj288:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:707a62d6:::obj465:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:707a62d6:::obj465:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:707a62d6:::obj465:head 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:707a62d6:::obj465:head v 107'465 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.571+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:707a62d6:::obj465:head, version: 107'465, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:707a62d6:::obj465:head@107'465, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:70360d67:::obj288:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.571+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:70360d67:::obj288:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 114/114 lb 4:70360d67:::obj288:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.571+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.571+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:70360d67:::obj288:head) v3 2026-02-11T23:31:17.571+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:6f5862a4:::obj430:head local-lis/les=112/113 n=260 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:70360d67:::obj288:head) 2026-02-11T23:31:17.571+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.571+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:70360d67:::obj288:head) v3 finish latency 0.000011 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.571+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.571+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d500 con 0x3e4a000 2026-02-11T23:31:17.571+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:707a62d6:::obj465:head)]) -- 0x4c2c000 con 0x3e4a000 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.571+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 570 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3e4a000 2026-02-11T23:31:17.571+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.571+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.571+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 571 ==== pg_backfill(progress 4.0 e 114/114 lb 4:707a62d6:::obj465:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3e4a000 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.571+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:707a62d6:::obj465:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.571+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:707a62d6:::obj465:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:707a62d6:::obj465:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7109478f:::obj25:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7109478f:::obj25:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7109478f:::obj25:head 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7109478f:::obj25:head v 107'25 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.571+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.571+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7109478f:::obj25:head, version: 107'25, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7109478f:::obj25:head@107'25, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.571+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:707a62d6:::obj465:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.571+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:707a62d6:::obj465:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 114/114 lb 4:707a62d6:::obj465:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.571+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.571+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:707a62d6:::obj465:head) v3 2026-02-11T23:31:17.571+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:70360d67:::obj288:head local-lis/les=112/113 n=261 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:707a62d6:::obj465:head) 2026-02-11T23:31:17.571+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.571+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:707a62d6:::obj465:head) v3 finish latency 0.000010 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.571+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.572+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.572+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.572+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8ef00 con 0x3e4a000 2026-02-11T23:31:17.572+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7109478f:::obj25:head)]) -- 0x4c2c300 con 0x3e4a000 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.572+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 572 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2000 con 0x3e4a000 2026-02-11T23:31:17.572+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.572+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.572+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.572+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.572+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.572+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 573 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7109478f:::obj25:head) ==== 1061+0+0 (unknown 0 0 0) 0x5165180 con 0x3e4a000 2026-02-11T23:31:17.572+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7109478f:::obj25:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.572+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.572+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7109478f:::obj25:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7109478f:::obj25:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.572+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.572+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.572+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.572+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.572+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7135fd38:::obj57:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.572+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7135fd38:::obj57:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.572+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7135fd38:::obj57:head 2026-02-11T23:31:17.572+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7135fd38:::obj57:head v 107'57 2026-02-11T23:31:17.572+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.572+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.572+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7135fd38:::obj57:head, version: 107'57, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7135fd38:::obj57:head@107'57, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.572+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7109478f:::obj25:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.572+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7109478f:::obj25:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 114/114 lb 4:7109478f:::obj25:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.572+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.572+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7109478f:::obj25:head) v3 2026-02-11T23:31:17.572+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:707a62d6:::obj465:head local-lis/les=112/113 n=262 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7109478f:::obj25:head) 2026-02-11T23:31:17.572+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.572+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7109478f:::obj25:head) v3 finish latency 0.000010 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.572+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.572+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.572+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8f500 con 0x3e4a000 2026-02-11T23:31:17.572+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7135fd38:::obj57:head)]) -- 0x4c2c480 con 0x3e4a000 2026-02-11T23:31:17.572+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.573+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 574 ==== MOSDPGPush(4.0 114/112 [PushOp(4:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.573+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.573+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.573+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 575 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7135fd38:::obj57:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164c00 con 0x3e4a000 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.573+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7135fd38:::obj57:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.573+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7135fd38:::obj57:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7135fd38:::obj57:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:72da0e1a:::obj147:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:72da0e1a:::obj147:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:72da0e1a:::obj147:head 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:72da0e1a:::obj147:head v 107'147 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.573+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:72da0e1a:::obj147:head, version: 107'147, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:72da0e1a:::obj147:head@107'147, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7135fd38:::obj57:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7135fd38:::obj57:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 114/114 lb 4:7135fd38:::obj57:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7135fd38:::obj57:head) v3 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7109478f:::obj25:head local-lis/les=112/113 n=263 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7135fd38:::obj57:head) 2026-02-11T23:31:17.573+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7135fd38:::obj57:head) v3 finish latency 0.000011 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.573+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1ed80 con 0x3e4a000 2026-02-11T23:31:17.573+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:72da0e1a:::obj147:head)]) -- 0x4c2c600 con 0x3e4a000 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.573+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 576 ==== MOSDPGPush(4.0 114/112 [PushOp(4:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.573+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.573+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.573+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 577 ==== pg_backfill(progress 4.0 e 114/114 lb 4:72da0e1a:::obj147:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3e4a000 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.573+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:72da0e1a:::obj147:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.573+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:72da0e1a:::obj147:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:72da0e1a:::obj147:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:73627f05:::obj80:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:73627f05:::obj80:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:73627f05:::obj80:head 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:73627f05:::obj80:head v 107'80 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.573+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.573+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:73627f05:::obj80:head, version: 107'80, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:73627f05:::obj80:head@107'80, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:72da0e1a:::obj147:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:72da0e1a:::obj147:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 114/114 lb 4:72da0e1a:::obj147:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:72da0e1a:::obj147:head) v3 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7135fd38:::obj57:head local-lis/les=112/113 n=264 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:72da0e1a:::obj147:head) 2026-02-11T23:31:17.573+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:72da0e1a:::obj147:head) v3 finish latency 0.000010 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.573+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.573+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.573+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4fcf200 con 0x3e4a000 2026-02-11T23:31:17.573+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:73627f05:::obj80:head)]) -- 0x4c2c900 con 0x3e4a000 2026-02-11T23:31:17.573+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.574+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 578 ==== MOSDPGPush(4.0 114/112 [PushOp(4:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.574+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.574+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.574+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 579 ==== pg_backfill(progress 4.0 e 114/114 lb 4:73627f05:::obj80:head) ==== 1061+0+0 (unknown 0 0 0) 0x4def180 con 0x3e4a000 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.574+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:73627f05:::obj80:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.574+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:73627f05:::obj80:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:73627f05:::obj80:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:737833f6:::obj391:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:737833f6:::obj391:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:737833f6:::obj391:head 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:737833f6:::obj391:head v 107'391 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.574+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:737833f6:::obj391:head, version: 107'391, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:737833f6:::obj391:head@107'391, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:73627f05:::obj80:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.574+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:73627f05:::obj80:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:73627f05:::obj80:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.574+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.574+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:73627f05:::obj80:head) v3 2026-02-11T23:31:17.574+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:72da0e1a:::obj147:head local-lis/les=112/113 n=265 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:73627f05:::obj80:head) 2026-02-11T23:31:17.574+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.574+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:73627f05:::obj80:head) v3 finish latency 0.000010 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.574+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.574+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.574+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:737833f6:::obj391:head)]) -- 0x4c2ca80 con 0x3e4a000 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.574+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 580 ==== MOSDPGPush(4.0 114/112 [PushOp(4:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.574+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.574+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.574+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 581 ==== pg_backfill(progress 4.0 e 114/114 lb 4:737833f6:::obj391:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd180 con 0x3e4a000 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.574+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:737833f6:::obj391:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.574+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:737833f6:::obj391:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000022 MOSDPGPush(4.0 114/112 [PushOp(4:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:737833f6:::obj391:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:744a8d59:::obj243:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:744a8d59:::obj243:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:744a8d59:::obj243:head 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:744a8d59:::obj243:head v 107'243 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.574+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.574+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:744a8d59:::obj243:head, version: 107'243, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:744a8d59:::obj243:head@107'243, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.574+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:737833f6:::obj391:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.574+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:737833f6:::obj391:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 114/114 lb 4:737833f6:::obj391:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.574+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.574+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:737833f6:::obj391:head) v3 2026-02-11T23:31:17.574+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:73627f05:::obj80:head local-lis/les=112/113 n=266 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:737833f6:::obj391:head) 2026-02-11T23:31:17.574+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.574+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:737833f6:::obj391:head) v3 finish latency 0.000011 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.574+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.575+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.575+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.575+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1fc80 con 0x3e4a000 2026-02-11T23:31:17.575+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:744a8d59:::obj243:head)]) -- 0x4c2cd80 con 0x3e4a000 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.575+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 582 ==== MOSDPGPush(4.0 114/112 [PushOp(4:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3e4a000 2026-02-11T23:31:17.575+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 114 2026-02-11T23:31:17.575+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.575+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.575+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 583 ==== pg_backfill(progress 4.0 e 114/114 lb 4:744a8d59:::obj243:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3e4a000 2026-02-11T23:31:17.575+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.575+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:744a8d59:::obj243:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.575+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:744a8d59:::obj243:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.575+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.575+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.575+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.575+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.575+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:744a8d59:::obj243:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.575+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.575+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:748dbcc6:::obj428:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.575+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:748dbcc6:::obj428:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.575+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:748dbcc6:::obj428:head 2026-02-11T23:31:17.575+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:748dbcc6:::obj428:head v 107'428 2026-02-11T23:31:17.575+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.575+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.575+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:748dbcc6:::obj428:head, version: 107'428, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:748dbcc6:::obj428:head@107'428, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.575+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:744a8d59:::obj243:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.575+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:744a8d59:::obj243:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:744a8d59:::obj243:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.575+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.575+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:744a8d59:::obj243:head) v3 2026-02-11T23:31:17.575+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:737833f6:::obj391:head local-lis/les=112/113 n=267 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:744a8d59:::obj243:head) 2026-02-11T23:31:17.575+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.575+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:744a8d59:::obj243:head) v3 finish latency 0.000011 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.575+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.575+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.575+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1f080 con 0x3e4a000 2026-02-11T23:31:17.575+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:748dbcc6:::obj428:head)]) -- 0x4c2d080 con 0x3e4a000 2026-02-11T23:31:17.575+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.576+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 584 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51831e0 con 0x3e4a000 2026-02-11T23:31:17.576+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 114 2026-02-11T23:31:17.576+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.576+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 585 ==== pg_backfill(progress 4.0 e 114/114 lb 4:748dbcc6:::obj428:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3e4a000 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.576+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:748dbcc6:::obj428:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.576+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:748dbcc6:::obj428:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:748dbcc6:::obj428:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7492d19a:::obj168:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7492d19a:::obj168:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7492d19a:::obj168:head 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7492d19a:::obj168:head v 107'168 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.576+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7492d19a:::obj168:head, version: 107'168, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7492d19a:::obj168:head@107'168, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:748dbcc6:::obj428:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:748dbcc6:::obj428:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:748dbcc6:::obj428:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:748dbcc6:::obj428:head) v3 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:744a8d59:::obj243:head local-lis/les=112/113 n=268 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:748dbcc6:::obj428:head) 2026-02-11T23:31:17.576+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:748dbcc6:::obj428:head) v3 finish latency 0.000010 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.576+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x511bc80 con 0x3e4a000 2026-02-11T23:31:17.576+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7492d19a:::obj168:head)]) -- 0x4c2d380 con 0x3e4a000 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.576+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 586 ==== MOSDPGPush(4.0 114/112 [PushOp(4:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3e4a000 2026-02-11T23:31:17.576+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.576+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.576+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 587 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7492d19a:::obj168:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3e4a000 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.576+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7492d19a:::obj168:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 114 2026-02-11T23:31:17.576+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7492d19a:::obj168:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7492d19a:::obj168:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:750d67aa:::obj572:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:750d67aa:::obj572:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:750d67aa:::obj572:head 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:750d67aa:::obj572:head v 107'572 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.576+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.576+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:750d67aa:::obj572:head, version: 107'572, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:750d67aa:::obj572:head@107'572, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7492d19a:::obj168:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7492d19a:::obj168:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:7492d19a:::obj168:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7492d19a:::obj168:head) v3 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:748dbcc6:::obj428:head local-lis/les=112/113 n=269 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7492d19a:::obj168:head) 2026-02-11T23:31:17.576+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7492d19a:::obj168:head) v3 finish latency 0.000010 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.576+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.576+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x49d7800 con 0x3e4a000 2026-02-11T23:31:17.576+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:750d67aa:::obj572:head)]) -- 0x4c2d680 con 0x3e4a000 2026-02-11T23:31:17.576+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.576+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.577+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 588 ==== MOSDPGPush(4.0 114/112 [PushOp(4:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51821a0 con 0x3e4a000 2026-02-11T23:31:17.577+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.577+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.577+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 589 ==== pg_backfill(progress 4.0 e 114/114 lb 4:750d67aa:::obj572:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3e4a000 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.577+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:750d67aa:::obj572:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 114 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.577+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:750d67aa:::obj572:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:750d67aa:::obj572:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:76c71d4e:::obj92:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:76c71d4e:::obj92:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:76c71d4e:::obj92:head 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:76c71d4e:::obj92:head v 107'92 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.577+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:76c71d4e:::obj92:head, version: 107'92, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76c71d4e:::obj92:head@107'92, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:750d67aa:::obj572:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:750d67aa:::obj572:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:750d67aa:::obj572:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:750d67aa:::obj572:head) v3 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7492d19a:::obj168:head local-lis/les=112/113 n=270 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:750d67aa:::obj572:head) 2026-02-11T23:31:17.577+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:750d67aa:::obj572:head) v3 finish latency 0.000011 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.577+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x511bb00 con 0x3e4a000 2026-02-11T23:31:17.577+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:76c71d4e:::obj92:head)]) -- 0x4c2d980 con 0x3e4a000 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.577+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 590 ==== MOSDPGPush(4.0 114/112 [PushOp(4:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3e4a000 2026-02-11T23:31:17.577+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.577+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.577+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 591 ==== pg_backfill(progress 4.0 e 114/114 lb 4:76c71d4e:::obj92:head) ==== 1061+0+0 (unknown 0 0 0) 0x4ded080 con 0x3e4a000 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.577+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:76c71d4e:::obj92:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.577+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:76c71d4e:::obj92:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:76c71d4e:::obj92:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:76f9746b:::obj110:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:76f9746b:::obj110:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:76f9746b:::obj110:head 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:76f9746b:::obj110:head v 107'110 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.577+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.577+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:76f9746b:::obj110:head, version: 107'110, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:76f9746b:::obj110:head@107'110, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:76c71d4e:::obj92:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:76c71d4e:::obj92:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 114/114 lb 4:76c71d4e:::obj92:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:76c71d4e:::obj92:head) v3 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:750d67aa:::obj572:head local-lis/les=112/113 n=271 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:76c71d4e:::obj92:head) 2026-02-11T23:31:17.577+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:76c71d4e:::obj92:head) v3 finish latency 0.000012 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.577+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.577+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.577+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.578+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1f380 con 0x3e4a000 2026-02-11T23:31:17.578+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:76f9746b:::obj110:head)]) -- 0x4c2db00 con 0x3e4a000 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.578+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 592 ==== MOSDPGPush(4.0 114/112 [PushOp(4:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3e4a000 2026-02-11T23:31:17.578+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.578+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.578+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.578+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 593 ==== pg_backfill(progress 4.0 e 114/114 lb 4:76f9746b:::obj110:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3e4a000 2026-02-11T23:31:17.578+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.578+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:76f9746b:::obj110:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.578+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:76f9746b:::obj110:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.578+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.578+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.578+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.578+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:76f9746b:::obj110:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.578+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.578+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.578+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:775ec905:::obj98:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.578+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:775ec905:::obj98:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.578+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:775ec905:::obj98:head 2026-02-11T23:31:17.578+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:775ec905:::obj98:head v 107'98 2026-02-11T23:31:17.578+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.578+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.578+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:775ec905:::obj98:head, version: 107'98, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:775ec905:::obj98:head@107'98, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.578+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:76f9746b:::obj110:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.578+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:76f9746b:::obj110:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:76f9746b:::obj110:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.578+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.578+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:76f9746b:::obj110:head) v3 2026-02-11T23:31:17.578+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76c71d4e:::obj92:head local-lis/les=112/113 n=272 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:76f9746b:::obj110:head) 2026-02-11T23:31:17.578+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.578+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:76f9746b:::obj110:head) v3 finish latency 0.000010 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.578+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.578+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.578+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.578+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:775ec905:::obj98:head)]) -- 0x4c2de00 con 0x3e4a000 2026-02-11T23:31:17.578+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.579+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 594 ==== MOSDPGPush(4.0 114/112 [PushOp(4:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.579+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.579+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.579+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 595 ==== pg_backfill(progress 4.0 e 114/114 lb 4:775ec905:::obj98:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3e4a000 2026-02-11T23:31:17.579+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:775ec905:::obj98:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.579+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:775ec905:::obj98:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 114/112 [PushOp(4:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:775ec905:::obj98:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:776b5b83:::obj306:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:776b5b83:::obj306:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:776b5b83:::obj306:head 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:776b5b83:::obj306:head v 107'306 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.579+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:776b5b83:::obj306:head, version: 107'306, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:776b5b83:::obj306:head@107'306, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:775ec905:::obj98:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:775ec905:::obj98:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 114/114 lb 4:775ec905:::obj98:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:775ec905:::obj98:head) v3 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:76f9746b:::obj110:head local-lis/les=112/113 n=273 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:775ec905:::obj98:head) 2026-02-11T23:31:17.579+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:775ec905:::obj98:head) v3 finish latency 0.000011 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.579+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d380 con 0x3e4a000 2026-02-11T23:31:17.579+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:776b5b83:::obj306:head)]) -- 0x4c1fc80 con 0x3e4a000 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.579+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 596 ==== MOSDPGPush(4.0 114/112 [PushOp(4:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3d40 con 0x3e4a000 2026-02-11T23:31:17.579+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.579+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.579+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 597 ==== pg_backfill(progress 4.0 e 114/114 lb 4:776b5b83:::obj306:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3e4a000 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.579+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:776b5b83:::obj306:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.579+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:776b5b83:::obj306:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:776b5b83:::obj306:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:779a6cde:::obj377:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:779a6cde:::obj377:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:779a6cde:::obj377:head 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:779a6cde:::obj377:head v 107'377 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.579+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.579+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:779a6cde:::obj377:head, version: 107'377, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:779a6cde:::obj377:head@107'377, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:776b5b83:::obj306:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:776b5b83:::obj306:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 114/114 lb 4:776b5b83:::obj306:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:776b5b83:::obj306:head) v3 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:775ec905:::obj98:head local-lis/les=112/113 n=274 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:776b5b83:::obj306:head) 2026-02-11T23:31:17.579+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:776b5b83:::obj306:head) v3 finish latency 0.000010 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.579+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.579+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.579+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x511b680 con 0x3e4a000 2026-02-11T23:31:17.579+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:779a6cde:::obj377:head)]) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.579+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.580+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 598 ==== MOSDPGPush(4.0 114/112 [PushOp(4:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.580+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.580+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.580+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 599 ==== pg_backfill(progress 4.0 e 114/114 lb 4:779a6cde:::obj377:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3e4a000 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.580+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:779a6cde:::obj377:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.580+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:779a6cde:::obj377:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:779a6cde:::obj377:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:786d07d6:::obj325:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:786d07d6:::obj325:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:786d07d6:::obj325:head 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:786d07d6:::obj325:head v 107'325 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.580+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:786d07d6:::obj325:head, version: 107'325, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:786d07d6:::obj325:head@107'325, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:779a6cde:::obj377:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.580+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:779a6cde:::obj377:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:779a6cde:::obj377:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.580+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.580+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:779a6cde:::obj377:head) v3 2026-02-11T23:31:17.580+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:776b5b83:::obj306:head local-lis/les=112/113 n=275 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:779a6cde:::obj377:head) 2026-02-11T23:31:17.580+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.580+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:779a6cde:::obj377:head) v3 finish latency 0.000011 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.580+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.580+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4c1f680 con 0x3e4a000 2026-02-11T23:31:17.580+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:786d07d6:::obj325:head)]) -- 0x4fcf200 con 0x3e4a000 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.580+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 600 ==== MOSDPGPush(4.0 114/112 [PushOp(4:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.580+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.580+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.580+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 601 ==== pg_backfill(progress 4.0 e 114/114 lb 4:786d07d6:::obj325:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3e4a000 2026-02-11T23:31:17.580+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:786d07d6:::obj325:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.580+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:786d07d6:::obj325:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 114/112 [PushOp(4:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:786d07d6:::obj325:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:78a5b78c:::obj388:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:78a5b78c:::obj388:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:78a5b78c:::obj388:head 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:78a5b78c:::obj388:head v 107'388 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.580+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.580+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:78a5b78c:::obj388:head, version: 107'388, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:78a5b78c:::obj388:head@107'388, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.580+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:786d07d6:::obj325:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.580+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:786d07d6:::obj325:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 114/114 lb 4:786d07d6:::obj325:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.580+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.580+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:786d07d6:::obj325:head) v3 2026-02-11T23:31:17.580+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:779a6cde:::obj377:head local-lis/les=112/113 n=276 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:786d07d6:::obj325:head) 2026-02-11T23:31:17.580+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.580+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:786d07d6:::obj325:head) v3 finish latency 0.000010 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.580+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.581+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.581+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.581+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x511b380 con 0x3e4a000 2026-02-11T23:31:17.581+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:78a5b78c:::obj388:head)]) -- 0x4c1ed80 con 0x3e4a000 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.581+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 602 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3e4a000 2026-02-11T23:31:17.581+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.581+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.581+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.581+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.581+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.581+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 603 ==== pg_backfill(progress 4.0 e 114/114 lb 4:78a5b78c:::obj388:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3e4a000 2026-02-11T23:31:17.581+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.581+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:78a5b78c:::obj388:head) v3 prio 15 type 95 cost 0 latency 0.000011 epoch 114 2026-02-11T23:31:17.581+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:78a5b78c:::obj388:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.581+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.581+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:78a5b78c:::obj388:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.581+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.581+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.581+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7903d8f8:::obj457:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.581+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7903d8f8:::obj457:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.581+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7903d8f8:::obj457:head 2026-02-11T23:31:17.581+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7903d8f8:::obj457:head v 107'457 2026-02-11T23:31:17.581+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.581+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.581+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7903d8f8:::obj457:head, version: 107'457, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7903d8f8:::obj457:head@107'457, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.581+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:78a5b78c:::obj388:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.581+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:78a5b78c:::obj388:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:78a5b78c:::obj388:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.581+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.581+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:78a5b78c:::obj388:head) v3 2026-02-11T23:31:17.581+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:786d07d6:::obj325:head local-lis/les=112/113 n=277 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:78a5b78c:::obj388:head) 2026-02-11T23:31:17.581+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.581+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:78a5b78c:::obj388:head) v3 finish latency 0.000010 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.581+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.581+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.581+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4d8ec00 con 0x3e4a000 2026-02-11T23:31:17.581+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7903d8f8:::obj457:head)]) -- 0x4d8f500 con 0x3e4a000 2026-02-11T23:31:17.581+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.582+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 604 ==== MOSDPGPush(4.0 114/112 [PushOp(4:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3e4a000 2026-02-11T23:31:17.582+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 114 2026-02-11T23:31:17.582+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.582+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 605 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7903d8f8:::obj457:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3e4a000 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.582+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7903d8f8:::obj457:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.582+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7903d8f8:::obj457:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7903d8f8:::obj457:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:79ec8b1f:::obj508:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:79ec8b1f:::obj508:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:79ec8b1f:::obj508:head 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:79ec8b1f:::obj508:head v 107'508 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.582+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:79ec8b1f:::obj508:head, version: 107'508, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:79ec8b1f:::obj508:head@107'508, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7903d8f8:::obj457:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7903d8f8:::obj457:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 114/114 lb 4:7903d8f8:::obj457:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7903d8f8:::obj457:head) v3 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:78a5b78c:::obj388:head local-lis/les=112/113 n=278 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7903d8f8:::obj457:head) 2026-02-11T23:31:17.582+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7903d8f8:::obj457:head) v3 finish latency 0.000010 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.582+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x511b080 con 0x3e4a000 2026-02-11T23:31:17.582+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:79ec8b1f:::obj508:head)]) -- 0x4d8ef00 con 0x3e4a000 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.582+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 606 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.582+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.582+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.582+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 607 ==== pg_backfill(progress 4.0 e 114/114 lb 4:79ec8b1f:::obj508:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3e4a000 2026-02-11T23:31:17.582+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:79ec8b1f:::obj508:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.582+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:79ec8b1f:::obj508:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:79ec8b1f:::obj508:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7a057f92:::obj247:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7a057f92:::obj247:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7a057f92:::obj247:head 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7a057f92:::obj247:head v 107'247 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.582+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a057f92:::obj247:head, version: 107'247, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a057f92:::obj247:head@107'247, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:79ec8b1f:::obj508:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:79ec8b1f:::obj508:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:79ec8b1f:::obj508:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:79ec8b1f:::obj508:head) v3 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7903d8f8:::obj457:head local-lis/les=112/113 n=279 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:79ec8b1f:::obj508:head) 2026-02-11T23:31:17.582+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.582+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:79ec8b1f:::obj508:head) v3 finish latency 0.000013 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.582+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.582+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.582+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4e5d680 con 0x3e4a000 2026-02-11T23:31:17.582+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7a057f92:::obj247:head)]) -- 0x4c1f980 con 0x3e4a000 2026-02-11T23:31:17.582+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.583+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 608 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3e4a000 2026-02-11T23:31:17.583+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.583+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.583+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.583+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 609 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7a057f92:::obj247:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3e4a000 2026-02-11T23:31:17.583+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.583+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a057f92:::obj247:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.583+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.583+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.583+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a057f92:::obj247:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.583+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.583+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.583+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a057f92:::obj247:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.583+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.583+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.583+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.583+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7a539578:::obj175:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.583+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7a539578:::obj175:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.583+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7a539578:::obj175:head 2026-02-11T23:31:17.583+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7a539578:::obj175:head v 107'175 2026-02-11T23:31:17.583+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.583+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.583+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a539578:::obj175:head, version: 107'175, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a539578:::obj175:head@107'175, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.583+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.583+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.583+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a057f92:::obj247:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.583+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a057f92:::obj247:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 114/114 lb 4:7a057f92:::obj247:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.583+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.583+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7a057f92:::obj247:head) v3 2026-02-11T23:31:17.583+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:79ec8b1f:::obj508:head local-lis/les=112/113 n=280 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7a057f92:::obj247:head) 2026-02-11T23:31:17.583+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.583+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a057f92:::obj247:head) v3 finish latency 0.000013 2026-02-11T23:31:17.583+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.583+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.583+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.583+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.583+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.583+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.583+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.583+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x511ac00 con 0x3e4a000 2026-02-11T23:31:17.583+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7a539578:::obj175:head)]) -- 0x4e5d500 con 0x3e4a000 2026-02-11T23:31:17.583+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.584+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 610 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.584+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 114 2026-02-11T23:31:17.584+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.584+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 611 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7a539578:::obj175:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3e4a000 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.584+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a539578:::obj175:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.584+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a539578:::obj175:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a539578:::obj175:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7a71687f:::obj592:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7a71687f:::obj592:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7a71687f:::obj592:head 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7a71687f:::obj592:head v 107'592 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.584+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a71687f:::obj592:head, version: 107'592, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a71687f:::obj592:head@107'592, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a539578:::obj175:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a539578:::obj175:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 114/114 lb 4:7a539578:::obj175:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7a539578:::obj175:head) v3 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a057f92:::obj247:head local-lis/les=112/113 n=281 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7a539578:::obj175:head) 2026-02-11T23:31:17.584+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a539578:::obj175:head) v3 finish latency 0.000011 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.584+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae4000 con 0x3e4a000 2026-02-11T23:31:17.584+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7a71687f:::obj592:head)]) -- 0x4c1ea80 con 0x3e4a000 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.584+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 612 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3e4a000 2026-02-11T23:31:17.584+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.584+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.584+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 613 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7a71687f:::obj592:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5080 con 0x3e4a000 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.584+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a71687f:::obj592:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.584+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a71687f:::obj592:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a71687f:::obj592:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7a94cb37:::obj345:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7a94cb37:::obj345:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7a94cb37:::obj345:head 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7a94cb37:::obj345:head v 107'345 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.584+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.584+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a94cb37:::obj345:head, version: 107'345, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a94cb37:::obj345:head@107'345, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a71687f:::obj592:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a71687f:::obj592:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 114/114 lb 4:7a71687f:::obj592:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7a71687f:::obj592:head) v3 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a539578:::obj175:head local-lis/les=112/113 n=282 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7a71687f:::obj592:head) 2026-02-11T23:31:17.584+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a71687f:::obj592:head) v3 finish latency 0.000011 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.584+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.584+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.584+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae4180 con 0x3e4a000 2026-02-11T23:31:17.584+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7a94cb37:::obj345:head)]) -- 0x4e5d080 con 0x3e4a000 2026-02-11T23:31:17.584+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.585+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 614 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3e4a000 2026-02-11T23:31:17.585+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.585+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.585+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 615 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7a94cb37:::obj345:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3e4a000 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.585+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a94cb37:::obj345:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.585+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a94cb37:::obj345:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a94cb37:::obj345:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7a95c0e7:::obj245:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7a95c0e7:::obj245:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7a95c0e7:::obj245:head 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7a95c0e7:::obj245:head v 107'245 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.585+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7a95c0e7:::obj245:head, version: 107'245, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7a95c0e7:::obj245:head@107'245, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a94cb37:::obj345:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.585+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a94cb37:::obj345:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 114/114 lb 4:7a94cb37:::obj345:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.585+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.585+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7a94cb37:::obj345:head) v3 2026-02-11T23:31:17.585+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a71687f:::obj592:head local-lis/les=112/113 n=283 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7a94cb37:::obj345:head) 2026-02-11T23:31:17.585+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.585+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a94cb37:::obj345:head) v3 finish latency 0.000011 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.585+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.585+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae4300 con 0x3e4a000 2026-02-11T23:31:17.585+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7a95c0e7:::obj245:head)]) -- 0x4d8f200 con 0x3e4a000 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.585+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 616 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.585+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.585+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.585+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 617 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7a95c0e7:::obj245:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.585+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a95c0e7:::obj245:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 114 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.585+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a95c0e7:::obj245:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.585+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a95c0e7:::obj245:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7ac8ba2d:::obj506:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.585+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7ac8ba2d:::obj506:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7ac8ba2d:::obj506:head 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7ac8ba2d:::obj506:head v 107'506 2026-02-11T23:31:17.585+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.585+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.586+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ac8ba2d:::obj506:head, version: 107'506, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ac8ba2d:::obj506:head@107'506, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.586+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7a95c0e7:::obj245:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a95c0e7:::obj245:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 114/114 lb 4:7a95c0e7:::obj245:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7a95c0e7:::obj245:head) v3 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a94cb37:::obj345:head local-lis/les=112/113 n=284 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7a95c0e7:::obj245:head) 2026-02-11T23:31:17.586+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7a95c0e7:::obj245:head) v3 finish latency 0.000009 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.586+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.586+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae4480 con 0x3e4a000 2026-02-11T23:31:17.586+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7ac8ba2d:::obj506:head)]) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.586+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 618 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.586+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.586+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.586+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.586+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.586+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.586+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.586+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 619 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7ac8ba2d:::obj506:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5600 con 0x3e4a000 2026-02-11T23:31:17.586+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ac8ba2d:::obj506:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.586+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ac8ba2d:::obj506:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.586+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.586+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.586+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ac8ba2d:::obj506:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.586+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.586+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7af66aa8:::obj163:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.586+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7af66aa8:::obj163:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.586+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7af66aa8:::obj163:head 2026-02-11T23:31:17.586+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7af66aa8:::obj163:head v 107'163 2026-02-11T23:31:17.586+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.586+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.586+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7af66aa8:::obj163:head, version: 107'163, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7af66aa8:::obj163:head@107'163, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.586+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ac8ba2d:::obj506:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ac8ba2d:::obj506:head) v3 prio 15 cost 0 latency 0.000058 pg_backfill(progress 4.0 e 114/114 lb 4:7ac8ba2d:::obj506:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7ac8ba2d:::obj506:head) v3 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7a95c0e7:::obj245:head local-lis/les=112/113 n=285 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7ac8ba2d:::obj506:head) 2026-02-11T23:31:17.586+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ac8ba2d:::obj506:head) v3 finish latency 0.000010 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.586+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.586+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.586+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae4600 con 0x3e4a000 2026-02-11T23:31:17.586+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7af66aa8:::obj163:head)]) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.586+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.587+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 620 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.587+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.587+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.587+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.587+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 621 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7af66aa8:::obj163:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.587+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.587+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7af66aa8:::obj163:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.587+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7af66aa8:::obj163:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.587+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.587+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.587+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.587+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000022 MOSDPGPush(4.0 114/112 [PushOp(4:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.587+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.587+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.587+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.587+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.587+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7af66aa8:::obj163:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.587+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7b326036:::obj411:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.587+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7b326036:::obj411:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.587+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7b326036:::obj411:head 2026-02-11T23:31:17.587+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7b326036:::obj411:head v 107'411 2026-02-11T23:31:17.587+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.587+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.587+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7b326036:::obj411:head, version: 107'411, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7b326036:::obj411:head@107'411, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.587+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.587+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.587+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7af66aa8:::obj163:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.587+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7af66aa8:::obj163:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 114/114 lb 4:7af66aa8:::obj163:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.587+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.587+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7af66aa8:::obj163:head) v3 2026-02-11T23:31:17.587+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ac8ba2d:::obj506:head local-lis/les=112/113 n=286 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7af66aa8:::obj163:head) 2026-02-11T23:31:17.587+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.587+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7af66aa8:::obj163:head) v3 finish latency 0.000010 2026-02-11T23:31:17.587+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.587+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.587+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.587+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.587+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.587+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.587+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.587+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae4780 con 0x3e4a000 2026-02-11T23:31:17.587+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7b326036:::obj411:head)]) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.587+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.588+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 622 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.588+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.588+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.588+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 623 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7b326036:::obj411:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.588+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7b326036:::obj411:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.588+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7b326036:::obj411:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7b326036:::obj411:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7c121b42:::obj116:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7c121b42:::obj116:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7c121b42:::obj116:head 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7c121b42:::obj116:head v 107'116 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.588+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c121b42:::obj116:head, version: 107'116, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c121b42:::obj116:head@107'116, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7b326036:::obj411:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.588+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7b326036:::obj411:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 114/114 lb 4:7b326036:::obj411:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.588+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.588+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7b326036:::obj411:head) v3 2026-02-11T23:31:17.588+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7af66aa8:::obj163:head local-lis/les=112/113 n=287 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7b326036:::obj411:head) 2026-02-11T23:31:17.588+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.588+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7b326036:::obj411:head) v3 finish latency 0.000012 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.588+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.588+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae4900 con 0x3e4a000 2026-02-11T23:31:17.588+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7c121b42:::obj116:head)]) -- 0x49b8c00 con 0x3e4a000 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.588+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 624 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3e4a000 2026-02-11T23:31:17.588+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 114 2026-02-11T23:31:17.588+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.588+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 625 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7c121b42:::obj116:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.588+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c121b42:::obj116:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.588+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c121b42:::obj116:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 114/112 [PushOp(4:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c121b42:::obj116:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7c3a2962:::obj277:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7c3a2962:::obj277:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7c3a2962:::obj277:head 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7c3a2962:::obj277:head v 107'277 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.588+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.588+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c3a2962:::obj277:head, version: 107'277, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c3a2962:::obj277:head@107'277, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.588+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c121b42:::obj116:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.588+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c121b42:::obj116:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 114/114 lb 4:7c121b42:::obj116:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.588+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.588+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7c121b42:::obj116:head) v3 2026-02-11T23:31:17.588+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7b326036:::obj411:head local-lis/les=112/113 n=288 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7c121b42:::obj116:head) 2026-02-11T23:31:17.588+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.588+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c121b42:::obj116:head) v3 finish latency 0.000014 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.588+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.589+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.589+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.589+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae4a80 con 0x3e4a000 2026-02-11T23:31:17.589+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7c3a2962:::obj277:head)]) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.589+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 626 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea1a0 con 0x3e4a000 2026-02-11T23:31:17.589+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.589+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.589+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.589+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 627 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7c3a2962:::obj277:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.589+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.589+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c3a2962:::obj277:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.589+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c3a2962:::obj277:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.589+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.589+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.589+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.589+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.589+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c3a2962:::obj277:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.589+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.589+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7c406887:::obj566:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.589+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7c406887:::obj566:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.589+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7c406887:::obj566:head 2026-02-11T23:31:17.589+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7c406887:::obj566:head v 107'566 2026-02-11T23:31:17.589+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.589+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.589+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c406887:::obj566:head, version: 107'566, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c406887:::obj566:head@107'566, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.589+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c3a2962:::obj277:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.589+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c3a2962:::obj277:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 114/114 lb 4:7c3a2962:::obj277:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.589+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.589+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7c3a2962:::obj277:head) v3 2026-02-11T23:31:17.589+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c121b42:::obj116:head local-lis/les=112/113 n=289 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7c3a2962:::obj277:head) 2026-02-11T23:31:17.589+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.589+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c3a2962:::obj277:head) v3 finish latency 0.000012 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.589+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.589+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.589+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae4c00 con 0x3e4a000 2026-02-11T23:31:17.589+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7c406887:::obj566:head)]) -- 0x4c1e300 con 0x3e4a000 2026-02-11T23:31:17.589+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.590+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 628 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3e4a000 2026-02-11T23:31:17.590+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.590+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.590+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 629 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7c406887:::obj566:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.590+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c406887:::obj566:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 114 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.590+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c406887:::obj566:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 114/112 [PushOp(4:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c406887:::obj566:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7c955dfc:::obj322:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7c955dfc:::obj322:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7c955dfc:::obj322:head 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7c955dfc:::obj322:head v 107'322 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.590+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7c955dfc:::obj322:head, version: 107'322, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7c955dfc:::obj322:head@107'322, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c406887:::obj566:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c406887:::obj566:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:7c406887:::obj566:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7c406887:::obj566:head) v3 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c3a2962:::obj277:head local-lis/les=112/113 n=290 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7c406887:::obj566:head) 2026-02-11T23:31:17.590+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c406887:::obj566:head) v3 finish latency 0.000011 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.590+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.590+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7c955dfc:::obj322:head)]) -- 0x4ae4180 con 0x3e4a000 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.590+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 630 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dd40 con 0x3e4a000 2026-02-11T23:31:17.590+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.590+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.590+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 631 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7c955dfc:::obj322:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.590+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c955dfc:::obj322:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.590+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c955dfc:::obj322:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 114/112 [PushOp(4:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c955dfc:::obj322:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7ccaea2b:::obj486:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7ccaea2b:::obj486:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7ccaea2b:::obj486:head 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7ccaea2b:::obj486:head v 107'486 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.590+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.590+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ccaea2b:::obj486:head, version: 107'486, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ccaea2b:::obj486:head@107'486, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7c955dfc:::obj322:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c955dfc:::obj322:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 114/114 lb 4:7c955dfc:::obj322:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7c955dfc:::obj322:head) v3 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c406887:::obj566:head local-lis/les=112/113 n=291 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7c955dfc:::obj322:head) 2026-02-11T23:31:17.590+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7c955dfc:::obj322:head) v3 finish latency 0.000011 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.590+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.590+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.590+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae4f00 con 0x3e4a000 2026-02-11T23:31:17.590+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7ccaea2b:::obj486:head)]) -- 0x4ae4000 con 0x3e4a000 2026-02-11T23:31:17.590+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.591+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 632 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183380 con 0x3e4a000 2026-02-11T23:31:17.591+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.591+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.591+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 633 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7ccaea2b:::obj486:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.591+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ccaea2b:::obj486:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 114 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.591+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ccaea2b:::obj486:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ccaea2b:::obj486:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7e5f2af9:::obj239:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7e5f2af9:::obj239:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7e5f2af9:::obj239:head 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7e5f2af9:::obj239:head v 107'239 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.591+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7e5f2af9:::obj239:head, version: 107'239, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7e5f2af9:::obj239:head@107'239, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ccaea2b:::obj486:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.591+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ccaea2b:::obj486:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 114/114 lb 4:7ccaea2b:::obj486:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.591+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.591+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7ccaea2b:::obj486:head) v3 2026-02-11T23:31:17.591+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7c955dfc:::obj322:head local-lis/les=112/113 n=292 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7ccaea2b:::obj486:head) 2026-02-11T23:31:17.591+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.591+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ccaea2b:::obj486:head) v3 finish latency 0.000011 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.591+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.591+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae5080 con 0x3e4a000 2026-02-11T23:31:17.591+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7e5f2af9:::obj239:head)]) -- 0x511ac00 con 0x3e4a000 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.591+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 634 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4de44e0 con 0x3e4a000 2026-02-11T23:31:17.591+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.591+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.591+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 635 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7e5f2af9:::obj239:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3e4a000 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.591+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7e5f2af9:::obj239:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.591+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7e5f2af9:::obj239:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 114/112 [PushOp(4:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7e5f2af9:::obj239:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7eacede5:::obj24:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7eacede5:::obj24:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7eacede5:::obj24:head 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7eacede5:::obj24:head v 107'24 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.591+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.591+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7eacede5:::obj24:head, version: 107'24, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7eacede5:::obj24:head@107'24, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.591+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.591+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7e5f2af9:::obj239:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.591+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7e5f2af9:::obj239:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 114/114 lb 4:7e5f2af9:::obj239:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.591+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.592+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7e5f2af9:::obj239:head) v3 2026-02-11T23:31:17.592+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ccaea2b:::obj486:head local-lis/les=112/113 n=293 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7e5f2af9:::obj239:head) 2026-02-11T23:31:17.592+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.592+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7e5f2af9:::obj239:head) v3 finish latency 0.000013 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.592+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.592+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.592+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae5200 con 0x3e4a000 2026-02-11T23:31:17.592+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7eacede5:::obj24:head)]) -- 0x4e5d680 con 0x3e4a000 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.592+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 636 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d040 con 0x3e4a000 2026-02-11T23:31:17.592+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.592+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.592+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.592+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 637 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7eacede5:::obj24:head) ==== 1061+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.592+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.592+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7eacede5:::obj24:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.592+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.592+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7eacede5:::obj24:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.592+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.592+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 114/112 [PushOp(4:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.592+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.592+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.592+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7eacede5:::obj24:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.592+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7ec5cfba:::obj597:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.592+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7ec5cfba:::obj597:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.592+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7ec5cfba:::obj597:head 2026-02-11T23:31:17.592+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7ec5cfba:::obj597:head v 107'597 2026-02-11T23:31:17.592+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.592+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.592+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ec5cfba:::obj597:head, version: 107'597, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ec5cfba:::obj597:head@107'597, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.592+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7eacede5:::obj24:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.592+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7eacede5:::obj24:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 114/114 lb 4:7eacede5:::obj24:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.592+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.592+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7eacede5:::obj24:head) v3 2026-02-11T23:31:17.592+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7e5f2af9:::obj239:head local-lis/les=112/113 n=294 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7eacede5:::obj24:head) 2026-02-11T23:31:17.592+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.592+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7eacede5:::obj24:head) v3 finish latency 0.000010 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.592+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.592+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.592+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae5380 con 0x3e4a000 2026-02-11T23:31:17.592+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7ec5cfba:::obj597:head)]) -- 0x4d8ef00 con 0x3e4a000 2026-02-11T23:31:17.592+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.593+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 638 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.593+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 114 2026-02-11T23:31:17.593+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.593+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 639 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7ec5cfba:::obj597:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3e4a000 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.593+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ec5cfba:::obj597:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.593+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ec5cfba:::obj597:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 114/112 [PushOp(4:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ec5cfba:::obj597:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7ee98ed8:::obj216:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7ee98ed8:::obj216:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7ee98ed8:::obj216:head 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7ee98ed8:::obj216:head v 107'216 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.593+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ee98ed8:::obj216:head, version: 107'216, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee98ed8:::obj216:head@107'216, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ec5cfba:::obj597:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ec5cfba:::obj597:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 114/114 lb 4:7ec5cfba:::obj597:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7ec5cfba:::obj597:head) v3 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7eacede5:::obj24:head local-lis/les=112/113 n=295 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7ec5cfba:::obj597:head) 2026-02-11T23:31:17.593+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ec5cfba:::obj597:head) v3 finish latency 0.000010 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.593+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae5500 con 0x3e4a000 2026-02-11T23:31:17.593+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7ee98ed8:::obj216:head)]) -- 0x4d8f500 con 0x3e4a000 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.593+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 640 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.593+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 114 2026-02-11T23:31:17.593+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.593+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 641 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7ee98ed8:::obj216:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bc100 con 0x3e4a000 2026-02-11T23:31:17.593+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ee98ed8:::obj216:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 114 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.593+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ee98ed8:::obj216:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 114/112 [PushOp(4:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ee98ed8:::obj216:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7ee9cafa:::obj105:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7ee9cafa:::obj105:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7ee9cafa:::obj105:head 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7ee9cafa:::obj105:head v 107'105 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.593+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7ee9cafa:::obj105:head, version: 107'105, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7ee9cafa:::obj105:head@107'105, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ee98ed8:::obj216:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ee98ed8:::obj216:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 114/114 lb 4:7ee98ed8:::obj216:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7ee98ed8:::obj216:head) v3 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ec5cfba:::obj597:head local-lis/les=112/113 n=296 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7ee98ed8:::obj216:head) 2026-02-11T23:31:17.593+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.593+0000 7fac51168640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ee98ed8:::obj216:head) v3 finish latency 0.000014 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.593+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.593+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.593+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae5980 con 0x3e4a000 2026-02-11T23:31:17.593+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7ee9cafa:::obj105:head)]) -- 0x4ae5680 con 0x3e4a000 2026-02-11T23:31:17.593+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 642 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.594+0000 7fac703fa640 15 osd.0 114 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 114 2026-02-11T23:31:17.594+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 643 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7ee9cafa:::obj105:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3e4a000 2026-02-11T23:31:17.594+0000 7fac703fa640 15 osd.0 114 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ee9cafa:::obj105:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 114 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.594+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ee9cafa:::obj105:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ee9cafa:::obj105:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000052 MOSDPGPush(4.0 114/112 [PushOp(4:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7f12fc77:::obj195:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7f12fc77:::obj195:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7f12fc77:::obj195:head 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7f12fc77:::obj195:head v 107'195 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.594+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 114 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7f12fc77:::obj195:head, version: 107'195, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f12fc77:::obj195:head@107'195, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000070 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7ee9cafa:::obj105:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ee9cafa:::obj105:head) v3 prio 15 cost 0 latency 0.000106 pg_backfill(progress 4.0 e 114/114 lb 4:7ee9cafa:::obj105:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 114 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 114 is not newer than session's projected_epoch of 114 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7ee9cafa:::obj105:head) v3 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee98ed8:::obj216:head local-lis/les=112/113 n=297 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7ee9cafa:::obj105:head) 2026-02-11T23:31:17.594+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 114 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7ee9cafa:::obj105:head) v3 finish latency 0.000022 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1244 ==== osd_map(115..115 src has 1..115) ==== 296+0+0 (unknown 0 0 0) 0x3941880 con 0x3752000 2026-02-11T23:31:17.594+0000 7fac606e4640 20 osd.0 114 OSD::ms_dispatch: osd_map(115..115 src has 1..115) 2026-02-11T23:31:17.594+0000 7fac606e4640 20 osd.0 114 _dispatch 0x3941880 osd_map(115..115 src has 1..115) 2026-02-11T23:31:17.594+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1245 ==== pg_info2(1.0 1.0 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) e115/115) ==== 1085+0+0 (unknown 0 0 0) 0x4e5aa00 con 0x3752000 2026-02-11T23:31:17.594+0000 7fac606e4640 3 osd.0 114 handle_osd_map epochs [115,115], i have 114, src has [1,115] 2026-02-11T23:31:17.594+0000 7fac606e4640 10 osd.0 114 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.594+0000 7fac606e4640 10 osd.0 114 handle_osd_map got inc map for epoch 115 2026-02-11T23:31:17.594+0000 7fac703fa640 15 osd.0 114 enqueue_peering_evt 1.0 epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) pg_lease(ru 0.000000000s ub 84.448272705s int 16.000000000s) 2026-02-11T23:31:17.594+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) pg_lease(ru 0.000000000s ub 84.448272705s int 16.000000000s)) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) pg_lease(ru 0.000000000s ub 84.448272705s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:17.594+0000 7fac606e4640 20 osd.0 114 got_full_map 115, nothing requested 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) pg_lease(ru 0.000000000s ub 84.448272705s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) pg 0x4a2a000 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq _add_slot_waiter 1.0 peering, item epoch is 115, will wait on OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) pg_lease(ru 0.000000000s ub 84.448272705s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac606e4640 20 osd.0 114 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:17.594+0000 7fac606e4640 20 osd.0 114 handle_osd_map pg_num_history pg_num_history(e115 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:31:17.594+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {115={}} 2026-02-11T23:31:17.594+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:17.594+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e115 maps [1~115] lci=[5,115] tlb=1) 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 114 get_con_osd_cluster to osd.4 from_epoch 114 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 114 get_nextmap_reserved map_reservations: {114=1} 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 114 release_map epoch: 114 2026-02-11T23:31:17.594+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e114/114) -- 0x4ae5b00 con 0x3e4a000 2026-02-11T23:31:17.594+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7f12fc77:::obj195:head)]) -- 0x4c1ed80 con 0x3e4a000 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac5e983640 10 osd.0 114 _committed_osd_maps 115..115 2026-02-11T23:31:17.594+0000 7fac5e983640 10 osd.0 114 advance to epoch 115 (<= last 115 <= newest_map 115) 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0 115 consume_map version 115 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0 114 await_reserved_maps epoch:115 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0 114 await_reserved_maps done 0 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0 115 consume_map 115 -- publish done 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x4946160 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0 115 identify_splits_and_merges 4.0 e114 to e115 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0 115 identify_splits_and_merges 4.0 e114 to e115 pg_nums {45=1} 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x25809a0 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0 115 identify_splits_and_merges 1.0 e114 to e115 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0 115 identify_splits_and_merges 1.0 e114 to e115 pg_nums {24=1} 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0 115 prune_pg_created 2026-02-11T23:31:17.594+0000 7fac5e983640 10 osd.0:0.consume_map 115 (was 114) 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 pending_peering first epoch 115 <= 115, requeueing 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 1.0 to_process <> waiting <> waiting_peering {115=} 2026-02-11T23:31:17.594+0000 7fac5e983640 10 osd.0:1.consume_map 115 (was 114) 2026-02-11T23:31:17.594+0000 7fac5e983640 10 osd.0:2.consume_map 115 (was 114) 2026-02-11T23:31:17.594+0000 7fac5e983640 10 osd.0:3.consume_map 115 (was 114) 2026-02-11T23:31:17.594+0000 7fac5e983640 10 osd.0:4.consume_map 115 (was 114) 2026-02-11T23:31:17.594+0000 7fac5e983640 10 osd.0:5.consume_map 115 (was 114) 2026-02-11T23:31:17.594+0000 7fac5e983640 10 osd.0:6.consume_map 115 (was 114) 2026-02-11T23:31:17.594+0000 7fac5e983640 10 osd.0:7.consume_map 115 (was 114) 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) pg_lease(ru 0.000000000s ub 84.448272705s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) pg_lease(ru 0.000000000s ub 84.448272705s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) pg 0x4a2a000 2026-02-11T23:31:17.594+0000 7fac5e983640 10 osd.0 115 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] handle_advance_map: 115 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 114 -> 115 2026-02-11T23:31:17.594+0000 7fac5e983640 15 osd.0 115 enqueue_peering_evt 4.0 epoch_sent: 115 epoch_requested: 115 NullEvt 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 114 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 NullEvt) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] state: Started advmap 2026-02-11T23:31:17.594+0000 7fac5e983640 15 osd.0 115 enqueue_peering_evt 1.0 epoch_sent: 115 epoch_requested: 115 NullEvt 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.594+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 NullEvt) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 NullEvt) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 115 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] handle_activate_map: epoch range: 115..115 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 NullEvt) class_id 2immediate prio 255 cost 10 e115) pg 0x4859000 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.594+0000 7fac5e983640 7 osd.0 115 activate_map version 115 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_advance_map: 115 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0:0.update_pg_epoch 4.0 114 -> 115 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 114 while current is 115 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 114 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4] -- 4/4 2026-02-11T23:31:17.594+0000 7fac55971640 20 osd.0 115 agent_entry empty queue 2026-02-11T23:31:17.594+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) pg_lease(ru 0.000000000s ub 84.448272705s int 16.000000000s) 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] state: Started advmap 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] state: got info from osd.1 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 115 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 84.448272705s int 16.000000000s) 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_activate_map: epoch range: 115..115 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.980914831s -> ruub 86.429191589s 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::proc_lease readable_until_ub now 86.429191589s 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 114 while current is 115 2026-02-11T23:31:17.594+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.594+0000 7fac51168640 5 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] exit Started/Stray 0.102600 5 0.000047 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 NullEvt 2026-02-11T23:31:17.594+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.594+0000 7fac51168640 5 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] enter Started/ReplicaActive 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac51168640 5 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 NullEvt) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown NOTIFY mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 107'600 -> 107'600 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:31:17.594+0000 7fac51168640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:31:17.594+0000 7fac51168640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] state: Activate Finished 2026-02-11T23:31:17.594+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 115 get_con_osd_cluster to osd.1 from_epoch 115 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6812/4244365097 map epoch 114 -> 115 (shared) 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 115 send_incremental_map epoch range: (115, 115] to v2:127.0.0.1:6812/4244365097 2026-02-11T23:31:17.594+0000 7fac51168640 10 osd.0 115 add_map_inc_bl 115 252 bytes 2026-02-11T23:31:17.594+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- osd_map(115..115 src has 1..115) -- 0x3940fc0 con 0x3752000 2026-02-11T23:31:17.594+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_notify2(1.0 1.0 (query:115 sent:115 1.0( v 107'600 (107'590,107'600] local-lis/les=109/110 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) ([103,113] all_participants={0, 1} intervals=[([109,113] acting={1})])) e115/115) -- 0x5152000 con 0x3752000 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 NullEvt) class_id 2immediate prio 255 cost 10 e115) pg 0x4a2a000 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 NullEvt 2026-02-11T23:31:17.594+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac55170640 15 osd.0 115 enqueue_peering_evt 1.0 epoch_sent: 115 epoch_requested: 115 ActivateCommitted from 115 processed at 115 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 ActivateCommitted from 115 processed at 115) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 ActivateCommitted from 115 processed at 115) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 ActivateCommitted from 115 processed at 115) class_id 2immediate prio 255 cost 10 e115) pg 0x4a2a000 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 ActivateCommitted from 115 processed at 115 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 unknown mbc={}] state: react 115 telling primary 2026-02-11T23:31:17.594+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.1 from_epoch 115 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.594+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.594+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_info2(1.0 1.0 1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/103 les/c/f=115/105/0 sis=114) e115/115) -- 0x4d50300 con 0x3752000 2026-02-11T23:31:17.594+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.594+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1246 ==== pg_lease(1.0 pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s) e115/115) ==== 58+0+0 (unknown 0 0 0) 0x4c1ed80 con 0x3752000 2026-02-11T23:31:17.594+0000 7fac703fa640 15 osd.0 115 enqueue_peering_evt 1.0 epoch_sent: 115 epoch_requested: 115 MLease epoch 115 from osd.1 pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s) 2026-02-11T23:31:17.594+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MLease epoch 115 from osd.1 pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s)) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.594+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MLease epoch 115 from osd.1 pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MLease epoch 115 from osd.1 pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) pg 0x4a2a000 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 MLease epoch 115 from osd.1 pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s) 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s) 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -1.980729818s -> ru 86.429008484s 2026-02-11T23:31:17.595+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 644 ==== MOSDPGPush(4.0 114/112 [PushOp(4:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until now 86.429008484s 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.980914831s -> ruub 86.429855347s 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 86.429855347s 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 115 get_con_osd_cluster to osd.1 from_epoch 115 2026-02-11T23:31:17.595+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 114/112 [PushOp(4:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000013 epoch 114 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.595+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_lease_ack(1.0 pg_lease_ack(ruub 84.448936462s) e115/115) -- 0x4c306e0 con 0x3752000 2026-02-11T23:31:17.595+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e114) 2026-02-11T23:31:17.595+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) queued 2026-02-11T23:31:17.595+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 645 ==== pg_backfill(progress 4.0 e 114/114 lb 4:7f12fc77:::obj195:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3e4a000 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.595+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 114/114 lb 4:7f12fc77:::obj195:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 114 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 114/112 [PushOp(4:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e114) pg 0x4859000 2026-02-11T23:31:17.595+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7f12fc77:::obj195:head) v3) class_id 0 prio 15 cost 0 e114) 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7f12fc77:::obj195:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) queued 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 114/112 [PushOp(4:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 114 -> 115 (shared) 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 115 send_incremental_map epoch range: (115, 115] to v2:127.0.0.1:6836/864727417 2026-02-11T23:31:17.595+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(115..115 src has 1..115) -- 0x46eac40 con 0x3e4a000 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 114/112 [PushOp(4:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.595+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1247 ==== pg_info2(1.0 1.0 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/103 les/c/f=115/105/0 sis=114) e115/115) ==== 1137+0+0 (unknown 0 0 0) 0x4d50300 con 0x3752000 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.595+0000 7fac703fa640 15 osd.0 115 enqueue_peering_evt 1.0 epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/103 les/c/f=115/105/0 sis=114) pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s) 2026-02-11T23:31:17.595+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/103 les/c/f=115/105/0 sis=114) pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s)) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7f1dde7e:::obj491:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7f1dde7e:::obj491:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7f1dde7e:::obj491:head 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7f1dde7e:::obj491:head v 107'491 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.595+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 114/112 [PushOp(4:7f1dde7e:::obj491:head, version: 107'491, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f1dde7e:::obj491:head@107'491, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000069 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/103 les/c/f=115/105/0 sis=114) pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/103 les/c/f=115/105/0 sis=114) pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) pg 0x4a2a000 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=109/103 les/c/f=110/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/103 les/c/f=115/105/0 sis=114) pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s) 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 114/114 lb 4:7f12fc77:::obj195:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e114) pg 0x4859000 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/103 les/c/f=115/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 active mbc={}] PeeringState::update_history advanced history from ec=24/24 lis/c=114/103 les/c/f=115/105/0 sis=114 2026-02-11T23:31:17.595+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.595+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7f12fc77:::obj195:head) v3 prio 15 cost 0 latency 0.000094 pg_backfill(progress 4.0 e 114/114 lb 4:7f12fc77:::obj195:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.595+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.595+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 114/114 lb 4:7f12fc77:::obj195:head) v3 2026-02-11T23:31:17.595+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7ee9cafa:::obj105:head local-lis/les=112/113 n=298 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 114/114 lb 4:7f12fc77:::obj195:head) 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.595+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.595+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 114/114 lb 4:7f12fc77:::obj195:head) v3 finish latency 0.000016 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.595+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1248 ==== pg_info2(1.0 1.0 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) e115/115) ==== 1137+0+0 (unknown 0 0 0) 0x4d50300 con 0x3752000 2026-02-11T23:31:17.595+0000 7fac703fa640 15 osd.0 115 enqueue_peering_evt 1.0 epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s) 2026-02-11T23:31:17.595+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s)) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) pg 0x4a2a000 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.595+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/103 les/c/f=115/105/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 MInfoRec from 1 info: 1.0( v 107'600 (107'500,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) pg_lease(ru 84.448272705s ub 84.448936462s int 16.000000000s) 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 active mbc={}] PeeringState::update_history advanced history from ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 pi=[103,114)/1 crt=107'600 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:31:17.595+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.595+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.595+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae5c80 con 0x3e4a000 2026-02-11T23:31:17.595+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 114/112 [PushReplyOp(4:7f1dde7e:::obj491:head)]) -- 0x4fcf200 con 0x3e4a000 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.595+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 646 ==== MOSDPGPush(4.0 115/112 [PushOp(4:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ebba0 con 0x3e4a000 2026-02-11T23:31:17.595+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.595+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.595+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7f2a7765:::obj252:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.595+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7f2a7765:::obj252:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7f2a7765:::obj252:head 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7f2a7765:::obj252:head v 107'252 2026-02-11T23:31:17.595+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.595+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.596+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 647 ==== osd_map(115..115 src has 1..115) ==== 296+0+0 (unknown 0 0 0) 0x46eac40 con 0x3e4a000 2026-02-11T23:31:17.596+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 648 ==== pg_backfill(progress 4.0 e 115/115 lb 4:7f1dde7e:::obj491:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3e4a000 2026-02-11T23:31:17.596+0000 7fac606e4640 20 osd.0 115 OSD::ms_dispatch: osd_map(115..115 src has 1..115) 2026-02-11T23:31:17.596+0000 7fac606e4640 20 osd.0 115 _dispatch 0x46eac40 osd_map(115..115 src has 1..115) 2026-02-11T23:31:17.596+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:7f1dde7e:::obj491:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.596+0000 7fac606e4640 3 osd.0 115 handle_osd_map epochs [115,115], i have 115, src has [1,115] 2026-02-11T23:31:17.596+0000 7fac606e4640 10 osd.0 115 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.596+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:7f1dde7e:::obj491:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.596+0000 7fac606e4640 10 osd.0 115 no new maps here, dropping 2026-02-11T23:31:17.596+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:7f2a7765:::obj252:head, version: 107'252, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7f2a7765:::obj252:head@107'252, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:7f1dde7e:::obj491:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:7f1dde7e:::obj491:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.596+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:7f1dde7e:::obj491:head) v3 prio 15 cost 0 latency 0.000017 pg_backfill(progress 4.0 e 115/115 lb 4:7f1dde7e:::obj491:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:7f1dde7e:::obj491:head) v3 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f12fc77:::obj195:head local-lis/les=112/113 n=299 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:7f1dde7e:::obj491:head) 2026-02-11T23:31:17.596+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:7f1dde7e:::obj491:head) v3 finish latency 0.000022 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.596+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.596+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae5e00 con 0x3e4a000 2026-02-11T23:31:17.596+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:7f2a7765:::obj252:head)]) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.596+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 649 ==== MOSDPGPush(4.0 115/112 [PushOp(4:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3e4a000 2026-02-11T23:31:17.596+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.596+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.596+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.596+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.596+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.596+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.596+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 650 ==== pg_backfill(progress 4.0 e 115/115 lb 4:7f2a7765:::obj252:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3e4a000 2026-02-11T23:31:17.596+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:7f2a7765:::obj252:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.596+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:7f2a7765:::obj252:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.596+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.596+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.596+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.596+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:7f2a7765:::obj252:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.596+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:7fffd9c5:::obj553:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.596+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:7fffd9c5:::obj553:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.596+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:7fffd9c5:::obj553:head 2026-02-11T23:31:17.596+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:7fffd9c5:::obj553:head v 107'553 2026-02-11T23:31:17.596+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.596+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.596+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:7fffd9c5:::obj553:head, version: 107'553, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:7fffd9c5:::obj553:head@107'553, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.596+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:7f2a7765:::obj252:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:7f2a7765:::obj252:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 4.0 e 115/115 lb 4:7f2a7765:::obj252:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:7f2a7765:::obj252:head) v3 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f1dde7e:::obj491:head local-lis/les=112/113 n=300 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:7f2a7765:::obj252:head) 2026-02-11T23:31:17.596+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:7f2a7765:::obj252:head) v3 finish latency 0.000012 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.596+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.596+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.596+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4fcf200 con 0x3e4a000 2026-02-11T23:31:17.596+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:7fffd9c5:::obj553:head)]) -- 0x4c1fc80 con 0x3e4a000 2026-02-11T23:31:17.596+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.597+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 651 ==== MOSDPGPush(4.0 115/112 [PushOp(4:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3e4a000 2026-02-11T23:31:17.597+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.597+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.597+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 652 ==== pg_backfill(progress 4.0 e 115/115 lb 4:7fffd9c5:::obj553:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3e4a000 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.597+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:7fffd9c5:::obj553:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 115 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.597+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:7fffd9c5:::obj553:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:7fffd9c5:::obj553:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:81617860:::obj28:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:81617860:::obj28:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:81617860:::obj28:head 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:81617860:::obj28:head v 107'28 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.597+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:81617860:::obj28:head, version: 107'28, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81617860:::obj28:head@107'28, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:7fffd9c5:::obj553:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:7fffd9c5:::obj553:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:7fffd9c5:::obj553:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:7fffd9c5:::obj553:head) v3 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7f2a7765:::obj252:head local-lis/les=112/113 n=301 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:7fffd9c5:::obj553:head) 2026-02-11T23:31:17.597+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:7fffd9c5:::obj553:head) v3 finish latency 0.000013 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.597+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae5c80 con 0x3e4a000 2026-02-11T23:31:17.597+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:81617860:::obj28:head)]) -- 0x4c2de00 con 0x3e4a000 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.597+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 653 ==== MOSDPGPush(4.0 115/112 [PushOp(4:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3e4a000 2026-02-11T23:31:17.597+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.597+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.597+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 654 ==== pg_backfill(progress 4.0 e 115/115 lb 4:81617860:::obj28:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3e4a000 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.597+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:81617860:::obj28:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.597+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:81617860:::obj28:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:81617860:::obj28:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:81fa42e1:::obj447:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:81fa42e1:::obj447:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:81fa42e1:::obj447:head 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:81fa42e1:::obj447:head v 107'447 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.597+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.597+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:81fa42e1:::obj447:head, version: 107'447, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:81fa42e1:::obj447:head@107'447, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:81617860:::obj28:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:81617860:::obj28:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 115/115 lb 4:81617860:::obj28:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:81617860:::obj28:head) v3 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:7fffd9c5:::obj553:head local-lis/les=112/113 n=302 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:81617860:::obj28:head) 2026-02-11T23:31:17.597+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:81617860:::obj28:head) v3 finish latency 0.000014 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.597+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.597+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.597+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.597+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c1ed80 con 0x3e4a000 2026-02-11T23:31:17.598+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:81fa42e1:::obj447:head)]) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.598+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.598+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 655 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3e4a000 2026-02-11T23:31:17.598+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.598+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.598+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.598+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.598+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 656 ==== pg_backfill(progress 4.0 e 115/115 lb 4:81fa42e1:::obj447:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3e4a000 2026-02-11T23:31:17.598+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.598+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:81fa42e1:::obj447:head) v3 prio 15 type 95 cost 0 latency 0.000009 epoch 115 2026-02-11T23:31:17.598+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.598+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:81fa42e1:::obj447:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.598+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.598+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:81fa42e1:::obj447:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.598+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 115/112 [PushOp(4:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.598+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.598+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.598+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.598+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8205b5d8:::obj400:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.598+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8205b5d8:::obj400:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.598+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8205b5d8:::obj400:head 2026-02-11T23:31:17.598+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8205b5d8:::obj400:head v 107'400 2026-02-11T23:31:17.598+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.598+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.598+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8205b5d8:::obj400:head, version: 107'400, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8205b5d8:::obj400:head@107'400, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.598+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.598+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.598+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:81fa42e1:::obj447:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.598+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:81fa42e1:::obj447:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 115/115 lb 4:81fa42e1:::obj447:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.598+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.598+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:81fa42e1:::obj447:head) v3 2026-02-11T23:31:17.598+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81617860:::obj28:head local-lis/les=112/113 n=303 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:81fa42e1:::obj447:head) 2026-02-11T23:31:17.598+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.598+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:81fa42e1:::obj447:head) v3 finish latency 0.000014 2026-02-11T23:31:17.598+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.598+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.598+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.598+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.598+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.598+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.598+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.598+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae5b00 con 0x3e4a000 2026-02-11T23:31:17.598+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8205b5d8:::obj400:head)]) -- 0x4c1f380 con 0x3e4a000 2026-02-11T23:31:17.598+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.599+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 657 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2340 con 0x3e4a000 2026-02-11T23:31:17.599+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.599+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.599+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 658 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8205b5d8:::obj400:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3e4a000 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.599+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8205b5d8:::obj400:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.599+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8205b5d8:::obj400:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8205b5d8:::obj400:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8250b831:::obj124:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8250b831:::obj124:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8250b831:::obj124:head 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8250b831:::obj124:head v 107'124 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.599+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8250b831:::obj124:head, version: 107'124, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8250b831:::obj124:head@107'124, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8205b5d8:::obj400:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8205b5d8:::obj400:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 115/115 lb 4:8205b5d8:::obj400:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8205b5d8:::obj400:head) v3 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:81fa42e1:::obj447:head local-lis/les=112/113 n=304 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8205b5d8:::obj400:head) 2026-02-11T23:31:17.599+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8205b5d8:::obj400:head) v3 finish latency 0.000013 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.599+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae5980 con 0x3e4a000 2026-02-11T23:31:17.599+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8250b831:::obj124:head)]) -- 0x511bb00 con 0x3e4a000 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.599+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 659 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.599+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.599+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.599+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 660 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8250b831:::obj124:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3e4a000 2026-02-11T23:31:17.599+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8250b831:::obj124:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 115 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.599+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8250b831:::obj124:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8250b831:::obj124:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8260b93f:::obj338:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8260b93f:::obj338:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8260b93f:::obj338:head 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8260b93f:::obj338:head v 107'338 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.599+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.599+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8260b93f:::obj338:head, version: 107'338, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8260b93f:::obj338:head@107'338, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8250b831:::obj124:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8250b831:::obj124:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:8250b831:::obj124:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8250b831:::obj124:head) v3 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8205b5d8:::obj400:head local-lis/les=112/113 n=305 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8250b831:::obj124:head) 2026-02-11T23:31:17.599+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8250b831:::obj124:head) v3 finish latency 0.000011 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.599+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.599+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.599+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4d8f500 con 0x3e4a000 2026-02-11T23:31:17.599+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8260b93f:::obj338:head)]) -- 0x49d7800 con 0x3e4a000 2026-02-11T23:31:17.599+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.600+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 661 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.600+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.600+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.600+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 662 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8260b93f:::obj338:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3e4a000 2026-02-11T23:31:17.600+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8260b93f:::obj338:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.600+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8260b93f:::obj338:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.600+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.600+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8260b93f:::obj338:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8280535c:::obj236:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8280535c:::obj236:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8280535c:::obj236:head 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8280535c:::obj236:head v 107'236 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.600+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8280535c:::obj236:head, version: 107'236, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8280535c:::obj236:head@107'236, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.600+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.600+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8260b93f:::obj338:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.600+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8260b93f:::obj338:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 115/115 lb 4:8260b93f:::obj338:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.600+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.600+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8260b93f:::obj338:head) v3 2026-02-11T23:31:17.600+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8250b831:::obj124:head local-lis/les=112/113 n=306 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8260b93f:::obj338:head) 2026-02-11T23:31:17.600+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.600+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8260b93f:::obj338:head) v3 finish latency 0.000016 2026-02-11T23:31:17.600+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.600+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.600+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae5380 con 0x3e4a000 2026-02-11T23:31:17.600+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8280535c:::obj236:head)]) -- 0x4ae5500 con 0x3e4a000 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.600+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 663 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.600+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.600+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.600+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 664 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8280535c:::obj236:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d700 con 0x3e4a000 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.600+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8280535c:::obj236:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 115 2026-02-11T23:31:17.600+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8280535c:::obj236:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.600+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.600+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.600+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.600+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8280535c:::obj236:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8421ae49:::obj196:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8421ae49:::obj196:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8421ae49:::obj196:head 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8421ae49:::obj196:head v 107'196 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.601+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8421ae49:::obj196:head, version: 107'196, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8421ae49:::obj196:head@107'196, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000094 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8280535c:::obj236:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8280535c:::obj236:head) v3 prio 15 cost 0 latency 0.000125 pg_backfill(progress 4.0 e 115/115 lb 4:8280535c:::obj236:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8280535c:::obj236:head) v3 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8260b93f:::obj338:head local-lis/les=112/113 n=307 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8280535c:::obj236:head) 2026-02-11T23:31:17.601+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8280535c:::obj236:head) v3 finish latency 0.000028 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.601+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x511ac00 con 0x3e4a000 2026-02-11T23:31:17.601+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8421ae49:::obj196:head)]) -- 0x4e5d680 con 0x3e4a000 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.601+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 665 ==== MOSDPGPush(4.0 115/112 [PushOp(4:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.601+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.601+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.601+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 666 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8421ae49:::obj196:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3e4a000 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.601+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8421ae49:::obj196:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 115 2026-02-11T23:31:17.601+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8421ae49:::obj196:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 115/112 [PushOp(4:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8421ae49:::obj196:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:84855744:::obj451:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:84855744:::obj451:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:84855744:::obj451:head 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:84855744:::obj451:head v 107'451 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.601+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.601+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:84855744:::obj451:head, version: 107'451, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84855744:::obj451:head@107'451, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000086 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8421ae49:::obj196:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8421ae49:::obj196:head) v3 prio 15 cost 0 latency 0.000106 pg_backfill(progress 4.0 e 115/115 lb 4:8421ae49:::obj196:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8421ae49:::obj196:head) v3 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8280535c:::obj236:head local-lis/les=112/113 n=308 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8421ae49:::obj196:head) 2026-02-11T23:31:17.601+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8421ae49:::obj196:head) v3 finish latency 0.000022 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.601+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.601+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.601+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae5080 con 0x3e4a000 2026-02-11T23:31:17.601+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:84855744:::obj451:head)]) -- 0x511bc80 con 0x3e4a000 2026-02-11T23:31:17.601+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.602+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 667 ==== MOSDPGPush(4.0 115/112 [PushOp(4:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x434d040 con 0x3e4a000 2026-02-11T23:31:17.602+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.602+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.602+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 668 ==== pg_backfill(progress 4.0 e 115/115 lb 4:84855744:::obj451:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3e4a000 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.602+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:84855744:::obj451:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 115 2026-02-11T23:31:17.602+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:84855744:::obj451:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:84855744:::obj451:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:84c1344d:::obj97:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:84c1344d:::obj97:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:84c1344d:::obj97:head 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:84c1344d:::obj97:head v 107'97 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.602+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:84c1344d:::obj97:head, version: 107'97, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:84c1344d:::obj97:head@107'97, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:84855744:::obj451:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.602+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:84855744:::obj451:head) v3 prio 15 cost 0 latency 0.000095 pg_backfill(progress 4.0 e 115/115 lb 4:84855744:::obj451:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.602+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.602+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:84855744:::obj451:head) v3 2026-02-11T23:31:17.602+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8421ae49:::obj196:head local-lis/les=112/113 n=309 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:84855744:::obj451:head) 2026-02-11T23:31:17.602+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.602+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:84855744:::obj451:head) v3 finish latency 0.000019 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.602+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.602+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4000 con 0x3e4a000 2026-02-11T23:31:17.602+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:84c1344d:::obj97:head)]) -- 0x4c1f080 con 0x3e4a000 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.602+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 669 ==== MOSDPGPush(4.0 115/112 [PushOp(4:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de44e0 con 0x3e4a000 2026-02-11T23:31:17.602+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.602+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.602+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 670 ==== pg_backfill(progress 4.0 e 115/115 lb 4:84c1344d:::obj97:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3e4a000 2026-02-11T23:31:17.602+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:84c1344d:::obj97:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 115 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.602+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:84c1344d:::obj97:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 115/112 [PushOp(4:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.602+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:84c1344d:::obj97:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:86bb8a56:::obj416:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.602+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:86bb8a56:::obj416:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:86bb8a56:::obj416:head 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:86bb8a56:::obj416:head v 107'416 2026-02-11T23:31:17.602+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.603+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.603+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:86bb8a56:::obj416:head, version: 107'416, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:86bb8a56:::obj416:head@107'416, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.603+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:84c1344d:::obj97:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:84c1344d:::obj97:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 4.0 e 115/115 lb 4:84c1344d:::obj97:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:84c1344d:::obj97:head) v3 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84855744:::obj451:head local-lis/les=112/113 n=310 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:84c1344d:::obj97:head) 2026-02-11T23:31:17.603+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:84c1344d:::obj97:head) v3 finish latency 0.000016 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.603+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.603+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4f00 con 0x3e4a000 2026-02-11T23:31:17.603+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:86bb8a56:::obj416:head)]) -- 0x4c2cd80 con 0x3e4a000 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.603+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 671 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183380 con 0x3e4a000 2026-02-11T23:31:17.603+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.603+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.603+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 672 ==== pg_backfill(progress 4.0 e 115/115 lb 4:86bb8a56:::obj416:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3e4a000 2026-02-11T23:31:17.603+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:86bb8a56:::obj416:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.603+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.603+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:86bb8a56:::obj416:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.603+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.603+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.603+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.603+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 115/112 [PushOp(4:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.603+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:86bb8a56:::obj416:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.603+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.603+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.603+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8745f65b:::obj476:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.603+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8745f65b:::obj476:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.603+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8745f65b:::obj476:head 2026-02-11T23:31:17.603+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8745f65b:::obj476:head v 107'476 2026-02-11T23:31:17.603+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.603+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.603+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8745f65b:::obj476:head, version: 107'476, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8745f65b:::obj476:head@107'476, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000073 2026-02-11T23:31:17.603+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:86bb8a56:::obj416:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:86bb8a56:::obj416:head) v3 prio 15 cost 0 latency 0.000098 pg_backfill(progress 4.0 e 115/115 lb 4:86bb8a56:::obj416:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:86bb8a56:::obj416:head) v3 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:84c1344d:::obj97:head local-lis/les=112/113 n=311 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:86bb8a56:::obj416:head) 2026-02-11T23:31:17.603+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:86bb8a56:::obj416:head) v3 finish latency 0.000018 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.603+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.603+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.603+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4180 con 0x3e4a000 2026-02-11T23:31:17.603+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8745f65b:::obj476:head)]) -- 0x3da3500 con 0x3e4a000 2026-02-11T23:31:17.603+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.604+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 673 ==== MOSDPGPush(4.0 115/112 [PushOp(4:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516dd40 con 0x3e4a000 2026-02-11T23:31:17.604+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.604+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.604+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.604+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 674 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8745f65b:::obj476:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162000 con 0x3e4a000 2026-02-11T23:31:17.604+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.604+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8745f65b:::obj476:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 115 2026-02-11T23:31:17.604+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8745f65b:::obj476:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.604+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.604+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 115/112 [PushOp(4:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8745f65b:::obj476:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:87eb7818:::obj91:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.604+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:87eb7818:::obj91:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:87eb7818:::obj91:head 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:87eb7818:::obj91:head v 107'91 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.604+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:87eb7818:::obj91:head, version: 107'91, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:87eb7818:::obj91:head@107'91, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000068 2026-02-11T23:31:17.604+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8745f65b:::obj476:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8745f65b:::obj476:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 4.0 e 115/115 lb 4:8745f65b:::obj476:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8745f65b:::obj476:head) v3 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:86bb8a56:::obj416:head local-lis/les=112/113 n=312 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8745f65b:::obj476:head) 2026-02-11T23:31:17.604+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8745f65b:::obj476:head) v3 finish latency 0.000017 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.604+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.604+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.604+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:87eb7818:::obj91:head)]) -- 0x3b1b680 con 0x3e4a000 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.604+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 675 ==== MOSDPGPush(4.0 115/112 [PushOp(4:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3e4a000 2026-02-11T23:31:17.604+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000020 epoch 115 2026-02-11T23:31:17.604+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.604+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 676 ==== pg_backfill(progress 4.0 e 115/115 lb 4:87eb7818:::obj91:head) ==== 1061+0+0 (unknown 0 0 0) 0x5162580 con 0x3e4a000 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.604+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:87eb7818:::obj91:head) v3 prio 15 type 95 cost 0 latency 0.000010 epoch 115 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.604+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:87eb7818:::obj91:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000051 MOSDPGPush(4.0 115/112 [PushOp(4:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.604+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.604+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:87eb7818:::obj91:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:881499a5:::obj460:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:881499a5:::obj460:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:881499a5:::obj460:head 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:881499a5:::obj460:head v 107'460 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.604+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.604+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:881499a5:::obj460:head, version: 107'460, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:881499a5:::obj460:head@107'460, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.604+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.604+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.604+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:87eb7818:::obj91:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:87eb7818:::obj91:head) v3 prio 15 cost 0 latency 0.000086 pg_backfill(progress 4.0 e 115/115 lb 4:87eb7818:::obj91:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:87eb7818:::obj91:head) v3 2026-02-11T23:31:17.604+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8745f65b:::obj476:head local-lis/les=112/113 n=313 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:87eb7818:::obj91:head) 2026-02-11T23:31:17.604+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.605+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:87eb7818:::obj91:head) v3 finish latency 0.000037 2026-02-11T23:31:17.605+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.605+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.605+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.605+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.605+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:881499a5:::obj460:head)]) -- 0x4c1e300 con 0x3e4a000 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.605+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.605+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 677 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea1a0 con 0x3e4a000 2026-02-11T23:31:17.605+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000014 epoch 115 2026-02-11T23:31:17.605+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.605+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.605+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 678 ==== pg_backfill(progress 4.0 e 115/115 lb 4:881499a5:::obj460:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3e4a000 2026-02-11T23:31:17.605+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.605+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:881499a5:::obj460:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 115 2026-02-11T23:31:17.605+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:881499a5:::obj460:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.605+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.605+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.605+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000041 MOSDPGPush(4.0 115/112 [PushOp(4:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.605+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.605+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:881499a5:::obj460:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.605+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.605+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8835249a:::obj346:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.605+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8835249a:::obj346:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.605+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8835249a:::obj346:head 2026-02-11T23:31:17.605+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8835249a:::obj346:head v 107'346 2026-02-11T23:31:17.605+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.605+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.605+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8835249a:::obj346:head, version: 107'346, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8835249a:::obj346:head@107'346, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.605+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:881499a5:::obj460:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.605+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:881499a5:::obj460:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 115/115 lb 4:881499a5:::obj460:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.605+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.605+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:881499a5:::obj460:head) v3 2026-02-11T23:31:17.605+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:87eb7818:::obj91:head local-lis/les=112/113 n=314 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:881499a5:::obj460:head) 2026-02-11T23:31:17.605+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.605+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:881499a5:::obj460:head) v3 finish latency 0.000013 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.605+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.605+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.605+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4a80 con 0x3e4a000 2026-02-11T23:31:17.605+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8835249a:::obj346:head)]) -- 0x4ba2d80 con 0x3e4a000 2026-02-11T23:31:17.605+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.606+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 679 ==== MOSDPGPush(4.0 115/112 [PushOp(4:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3e4a000 2026-02-11T23:31:17.606+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 115 2026-02-11T23:31:17.606+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.606+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 680 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8835249a:::obj346:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163080 con 0x3e4a000 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.606+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8835249a:::obj346:head) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 115 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.606+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8835249a:::obj346:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8835249a:::obj346:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:88c90a82:::obj574:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:88c90a82:::obj574:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:88c90a82:::obj574:head 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:88c90a82:::obj574:head v 107'574 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.606+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:88c90a82:::obj574:head, version: 107'574, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88c90a82:::obj574:head@107'574, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8835249a:::obj346:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.606+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8835249a:::obj346:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:8835249a:::obj346:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.606+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.606+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8835249a:::obj346:head) v3 2026-02-11T23:31:17.606+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:881499a5:::obj460:head local-lis/les=112/113 n=315 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8835249a:::obj346:head) 2026-02-11T23:31:17.606+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.606+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8835249a:::obj346:head) v3 finish latency 0.000012 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.606+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.606+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x49b8c00 con 0x3e4a000 2026-02-11T23:31:17.606+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:88c90a82:::obj574:head)]) -- 0x4f50180 con 0x3e4a000 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.606+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 681 ==== MOSDPGPush(4.0 115/112 [PushOp(4:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.606+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 115 2026-02-11T23:31:17.606+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.606+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 682 ==== pg_backfill(progress 4.0 e 115/115 lb 4:88c90a82:::obj574:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163600 con 0x3e4a000 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.606+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:88c90a82:::obj574:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.606+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:88c90a82:::obj574:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:88c90a82:::obj574:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:88e6b0fd:::obj99:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:88e6b0fd:::obj99:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:88e6b0fd:::obj99:head 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:88e6b0fd:::obj99:head v 107'99 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.606+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.606+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:88e6b0fd:::obj99:head, version: 107'99, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:88e6b0fd:::obj99:head@107'99, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.606+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:88c90a82:::obj574:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.606+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:88c90a82:::obj574:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:88c90a82:::obj574:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.606+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.606+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:88c90a82:::obj574:head) v3 2026-02-11T23:31:17.606+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8835249a:::obj346:head local-lis/les=112/113 n=316 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:88c90a82:::obj574:head) 2026-02-11T23:31:17.606+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.606+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:88c90a82:::obj574:head) v3 finish latency 0.000012 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.606+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.607+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.607+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.607+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4900 con 0x3e4a000 2026-02-11T23:31:17.607+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:88e6b0fd:::obj99:head)]) -- 0x4be8180 con 0x3e4a000 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.607+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 683 ==== MOSDPGPush(4.0 115/112 [PushOp(4:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.607+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.607+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.607+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.607+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 684 ==== pg_backfill(progress 4.0 e 115/115 lb 4:88e6b0fd:::obj99:head) ==== 1061+0+0 (unknown 0 0 0) 0x5163b80 con 0x3e4a000 2026-02-11T23:31:17.607+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.607+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:88e6b0fd:::obj99:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.607+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:88e6b0fd:::obj99:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.607+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.607+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.607+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.607+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:88e6b0fd:::obj99:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.607+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.607+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.607+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:89565a01:::obj330:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.607+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:89565a01:::obj330:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.607+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:89565a01:::obj330:head 2026-02-11T23:31:17.607+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:89565a01:::obj330:head v 107'330 2026-02-11T23:31:17.607+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.607+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.607+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:89565a01:::obj330:head, version: 107'330, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:89565a01:::obj330:head@107'330, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.607+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:88e6b0fd:::obj99:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.607+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:88e6b0fd:::obj99:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:88e6b0fd:::obj99:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.607+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.607+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:88e6b0fd:::obj99:head) v3 2026-02-11T23:31:17.607+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88c90a82:::obj574:head local-lis/les=112/113 n=317 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:88e6b0fd:::obj99:head) 2026-02-11T23:31:17.607+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.607+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:88e6b0fd:::obj99:head) v3 finish latency 0.000011 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.607+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.607+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.607+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.607+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:89565a01:::obj330:head)]) -- 0x4be8300 con 0x3e4a000 2026-02-11T23:31:17.607+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.608+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 685 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.608+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 115 2026-02-11T23:31:17.608+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.608+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 686 ==== pg_backfill(progress 4.0 e 115/115 lb 4:89565a01:::obj330:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.608+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:89565a01:::obj330:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.608+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:89565a01:::obj330:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:89565a01:::obj330:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8a5b4a06:::obj393:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8a5b4a06:::obj393:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8a5b4a06:::obj393:head 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8a5b4a06:::obj393:head v 107'393 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.608+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8a5b4a06:::obj393:head, version: 107'393, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a5b4a06:::obj393:head@107'393, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:89565a01:::obj330:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:89565a01:::obj330:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 115/115 lb 4:89565a01:::obj330:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:89565a01:::obj330:head) v3 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:88e6b0fd:::obj99:head local-lis/les=112/113 n=318 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:89565a01:::obj330:head) 2026-02-11T23:31:17.608+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:89565a01:::obj330:head) v3 finish latency 0.000012 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.608+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4780 con 0x3e4a000 2026-02-11T23:31:17.608+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8a5b4a06:::obj393:head)]) -- 0x4be8600 con 0x3e4a000 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.608+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 687 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.608+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.608+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.608+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 688 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8a5b4a06:::obj393:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.608+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8a5b4a06:::obj393:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.608+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8a5b4a06:::obj393:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8a5b4a06:::obj393:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8a7d5b8a:::obj217:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8a7d5b8a:::obj217:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8a7d5b8a:::obj217:head 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8a7d5b8a:::obj217:head v 107'217 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.608+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.608+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8a7d5b8a:::obj217:head, version: 107'217, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8a7d5b8a:::obj217:head@107'217, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8a5b4a06:::obj393:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8a5b4a06:::obj393:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 115/115 lb 4:8a5b4a06:::obj393:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8a5b4a06:::obj393:head) v3 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:89565a01:::obj330:head local-lis/les=112/113 n=319 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8a5b4a06:::obj393:head) 2026-02-11T23:31:17.608+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8a5b4a06:::obj393:head) v3 finish latency 0.000012 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.608+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.608+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.608+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.608+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8a7d5b8a:::obj217:head)]) -- 0x4be8900 con 0x3e4a000 2026-02-11T23:31:17.608+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.609+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 689 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a21a0 con 0x3e4a000 2026-02-11T23:31:17.609+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.609+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.609+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 690 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8a7d5b8a:::obj217:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.609+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8a7d5b8a:::obj217:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.609+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8a7d5b8a:::obj217:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8a7d5b8a:::obj217:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8aa18807:::obj353:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8aa18807:::obj353:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8aa18807:::obj353:head 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8aa18807:::obj353:head v 107'353 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.609+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8aa18807:::obj353:head, version: 107'353, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8aa18807:::obj353:head@107'353, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8a7d5b8a:::obj217:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.609+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8a7d5b8a:::obj217:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:8a7d5b8a:::obj217:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.609+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.609+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8a7d5b8a:::obj217:head) v3 2026-02-11T23:31:17.609+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a5b4a06:::obj393:head local-lis/les=112/113 n=320 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8a7d5b8a:::obj217:head) 2026-02-11T23:31:17.609+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.609+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8a7d5b8a:::obj217:head) v3 finish latency 0.000011 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.609+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.609+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4600 con 0x3e4a000 2026-02-11T23:31:17.609+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8aa18807:::obj353:head)]) -- 0x4be8c00 con 0x3e4a000 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.609+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 691 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2680 con 0x3e4a000 2026-02-11T23:31:17.609+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.609+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.609+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 692 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8aa18807:::obj353:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.609+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8aa18807:::obj353:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.609+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8aa18807:::obj353:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8aa18807:::obj353:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8b0915dc:::obj278:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8b0915dc:::obj278:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8b0915dc:::obj278:head 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8b0915dc:::obj278:head v 107'278 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.609+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.609+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b0915dc:::obj278:head, version: 107'278, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b0915dc:::obj278:head@107'278, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.609+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8aa18807:::obj353:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.609+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8aa18807:::obj353:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 115/115 lb 4:8aa18807:::obj353:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.609+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.609+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8aa18807:::obj353:head) v3 2026-02-11T23:31:17.609+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8a7d5b8a:::obj217:head local-lis/les=112/113 n=321 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8aa18807:::obj353:head) 2026-02-11T23:31:17.609+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.609+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8aa18807:::obj353:head) v3 finish latency 0.000011 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.609+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.610+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.610+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.610+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.610+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8b0915dc:::obj278:head)]) -- 0x4be8f00 con 0x3e4a000 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.610+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 693 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2b60 con 0x3e4a000 2026-02-11T23:31:17.610+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.610+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.610+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.610+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 694 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8b0915dc:::obj278:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.610+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.610+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b0915dc:::obj278:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.610+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b0915dc:::obj278:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.610+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.610+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.610+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.610+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.610+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.610+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b0915dc:::obj278:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.610+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8b38e8a9:::obj385:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.610+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8b38e8a9:::obj385:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.610+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8b38e8a9:::obj385:head 2026-02-11T23:31:17.610+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8b38e8a9:::obj385:head v 107'385 2026-02-11T23:31:17.610+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.610+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.610+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b38e8a9:::obj385:head, version: 107'385, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b38e8a9:::obj385:head@107'385, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.610+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b0915dc:::obj278:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.610+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b0915dc:::obj278:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 115/115 lb 4:8b0915dc:::obj278:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.610+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.610+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8b0915dc:::obj278:head) v3 2026-02-11T23:31:17.610+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8aa18807:::obj353:head local-lis/les=112/113 n=322 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8b0915dc:::obj278:head) 2026-02-11T23:31:17.610+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.610+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b0915dc:::obj278:head) v3 finish latency 0.000012 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.610+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.610+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.610+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4480 con 0x3e4a000 2026-02-11T23:31:17.610+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8b38e8a9:::obj385:head)]) -- 0x4be9200 con 0x3e4a000 2026-02-11T23:31:17.610+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.611+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 695 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dfe1a0 con 0x3e4a000 2026-02-11T23:31:17.611+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.611+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.611+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 696 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8b38e8a9:::obj385:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.611+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b38e8a9:::obj385:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.611+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b38e8a9:::obj385:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b38e8a9:::obj385:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8b716ab6:::obj324:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8b716ab6:::obj324:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8b716ab6:::obj324:head 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8b716ab6:::obj324:head v 107'324 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.611+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b716ab6:::obj324:head, version: 107'324, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b716ab6:::obj324:head@107'324, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b38e8a9:::obj385:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b38e8a9:::obj385:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:8b38e8a9:::obj385:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8b38e8a9:::obj385:head) v3 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b0915dc:::obj278:head local-lis/les=112/113 n=323 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8b38e8a9:::obj385:head) 2026-02-11T23:31:17.611+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b38e8a9:::obj385:head) v3 finish latency 0.000012 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.611+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4d8f200 con 0x3e4a000 2026-02-11T23:31:17.611+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8b716ab6:::obj324:head)]) -- 0x4be9500 con 0x3e4a000 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.611+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 697 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3520 con 0x3e4a000 2026-02-11T23:31:17.611+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.611+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.611+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 698 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8b716ab6:::obj324:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.611+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b716ab6:::obj324:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.611+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b716ab6:::obj324:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b716ab6:::obj324:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8b7a7bb6:::obj294:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8b7a7bb6:::obj294:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8b7a7bb6:::obj294:head 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8b7a7bb6:::obj294:head v 107'294 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.611+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.611+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8b7a7bb6:::obj294:head, version: 107'294, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8b7a7bb6:::obj294:head@107'294, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b716ab6:::obj324:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b716ab6:::obj324:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 115/115 lb 4:8b716ab6:::obj324:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8b716ab6:::obj324:head) v3 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b38e8a9:::obj385:head local-lis/les=112/113 n=324 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8b716ab6:::obj324:head) 2026-02-11T23:31:17.611+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b716ab6:::obj324:head) v3 finish latency 0.000011 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.611+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.611+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.611+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4300 con 0x3e4a000 2026-02-11T23:31:17.611+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8b7a7bb6:::obj294:head)]) -- 0x4be9800 con 0x3e4a000 2026-02-11T23:31:17.611+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.612+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 699 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183d40 con 0x3e4a000 2026-02-11T23:31:17.612+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.612+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.612+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 700 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8b7a7bb6:::obj294:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5600 con 0x3e4a000 2026-02-11T23:31:17.612+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b7a7bb6:::obj294:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.612+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b7a7bb6:::obj294:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b7a7bb6:::obj294:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8c123081:::obj473:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8c123081:::obj473:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8c123081:::obj473:head 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8c123081:::obj473:head v 107'473 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.612+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8c123081:::obj473:head, version: 107'473, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c123081:::obj473:head@107'473, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8b7a7bb6:::obj294:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.612+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b7a7bb6:::obj294:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 115/115 lb 4:8b7a7bb6:::obj294:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.612+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.612+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8b7a7bb6:::obj294:head) v3 2026-02-11T23:31:17.612+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b716ab6:::obj324:head local-lis/les=112/113 n=325 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8b7a7bb6:::obj294:head) 2026-02-11T23:31:17.612+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.612+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8b7a7bb6:::obj294:head) v3 finish latency 0.000011 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.612+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.612+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4e5d080 con 0x3e4a000 2026-02-11T23:31:17.612+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8c123081:::obj473:head)]) -- 0x4be9b00 con 0x3e4a000 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.612+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 701 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3e4a000 2026-02-11T23:31:17.612+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.612+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.612+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 702 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8c123081:::obj473:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.612+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8c123081:::obj473:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.612+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8c123081:::obj473:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8c123081:::obj473:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8c741e19:::obj599:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8c741e19:::obj599:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8c741e19:::obj599:head 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8c741e19:::obj599:head v 107'599 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.612+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.612+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8c741e19:::obj599:head, version: 107'599, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8c741e19:::obj599:head@107'599, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8c123081:::obj473:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.612+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8c123081:::obj473:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 4.0 e 115/115 lb 4:8c123081:::obj473:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.612+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.612+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8c123081:::obj473:head) v3 2026-02-11T23:31:17.612+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8b7a7bb6:::obj294:head local-lis/les=112/113 n=326 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8c123081:::obj473:head) 2026-02-11T23:31:17.612+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.612+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8c123081:::obj473:head) v3 finish latency 0.000010 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.612+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.612+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.613+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.613+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0000 con 0x3e4a000 2026-02-11T23:31:17.613+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8c741e19:::obj599:head)]) -- 0x4be9e00 con 0x3e4a000 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.613+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 703 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.613+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.613+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.613+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.613+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.613+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 704 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8c741e19:::obj599:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3e4a000 2026-02-11T23:31:17.613+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8c741e19:::obj599:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.613+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.613+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8c741e19:::obj599:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.613+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.613+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.613+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.613+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8c741e19:::obj599:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.613+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.613+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8ca4bb13:::obj342:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.613+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8ca4bb13:::obj342:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.613+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8ca4bb13:::obj342:head 2026-02-11T23:31:17.613+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8ca4bb13:::obj342:head v 107'342 2026-02-11T23:31:17.613+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.613+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.613+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8ca4bb13:::obj342:head, version: 107'342, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8ca4bb13:::obj342:head@107'342, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.613+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8c741e19:::obj599:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.613+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8c741e19:::obj599:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 115/115 lb 4:8c741e19:::obj599:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.613+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.613+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8c741e19:::obj599:head) v3 2026-02-11T23:31:17.613+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c123081:::obj473:head local-lis/les=112/113 n=327 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8c741e19:::obj599:head) 2026-02-11T23:31:17.613+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.613+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8c741e19:::obj599:head) v3 finish latency 0.000011 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.613+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.613+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.613+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0180 con 0x3e4a000 2026-02-11T23:31:17.613+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8ca4bb13:::obj342:head)]) -- 0x4ba2d80 con 0x3e4a000 2026-02-11T23:31:17.613+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.613+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 705 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.613+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.613+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.613+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.614+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 706 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8ca4bb13:::obj342:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5080 con 0x3e4a000 2026-02-11T23:31:17.614+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8ca4bb13:::obj342:head) v3 prio 15 type 95 cost 0 latency 0.000008 epoch 115 2026-02-11T23:31:17.614+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8ca4bb13:::obj342:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8ca4bb13:::obj342:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8cdc434f:::obj77:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8cdc434f:::obj77:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8cdc434f:::obj77:head 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8cdc434f:::obj77:head v 107'77 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.614+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8cdc434f:::obj77:head, version: 107'77, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8cdc434f:::obj77:head@107'77, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8ca4bb13:::obj342:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8ca4bb13:::obj342:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:8ca4bb13:::obj342:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8ca4bb13:::obj342:head) v3 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8c741e19:::obj599:head local-lis/les=112/113 n=328 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8ca4bb13:::obj342:head) 2026-02-11T23:31:17.614+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8ca4bb13:::obj342:head) v3 finish latency 0.000014 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.614+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0300 con 0x3e4a000 2026-02-11T23:31:17.614+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8cdc434f:::obj77:head)]) -- 0x4c1e300 con 0x3e4a000 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.614+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 707 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3a00 con 0x3e4a000 2026-02-11T23:31:17.614+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.614+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.614+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 708 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8cdc434f:::obj77:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3e4a000 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.614+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8cdc434f:::obj77:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.614+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8cdc434f:::obj77:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8cdc434f:::obj77:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8d29890f:::obj305:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8d29890f:::obj305:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8d29890f:::obj305:head 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8d29890f:::obj305:head v 107'305 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.614+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.614+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8d29890f:::obj305:head, version: 107'305, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d29890f:::obj305:head@107'305, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8cdc434f:::obj77:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8cdc434f:::obj77:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 115/115 lb 4:8cdc434f:::obj77:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8cdc434f:::obj77:head) v3 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8ca4bb13:::obj342:head local-lis/les=112/113 n=329 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8cdc434f:::obj77:head) 2026-02-11T23:31:17.614+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8cdc434f:::obj77:head) v3 finish latency 0.000010 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.614+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.614+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.614+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0480 con 0x3e4a000 2026-02-11T23:31:17.614+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8d29890f:::obj305:head)]) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.614+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.615+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 709 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.615+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.615+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.615+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 710 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8d29890f:::obj305:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3e4a000 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.615+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8d29890f:::obj305:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.615+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8d29890f:::obj305:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8d29890f:::obj305:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8d384ce8:::obj297:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8d384ce8:::obj297:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8d384ce8:::obj297:head 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8d384ce8:::obj297:head v 107'297 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.615+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8d384ce8:::obj297:head, version: 107'297, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d384ce8:::obj297:head@107'297, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8d29890f:::obj305:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8d29890f:::obj305:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 115/115 lb 4:8d29890f:::obj305:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8d29890f:::obj305:head) v3 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8cdc434f:::obj77:head local-lis/les=112/113 n=330 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8d29890f:::obj305:head) 2026-02-11T23:31:17.615+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8d29890f:::obj305:head) v3 finish latency 0.000010 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.615+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0600 con 0x3e4a000 2026-02-11T23:31:17.615+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8d384ce8:::obj297:head)]) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.615+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 711 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3e4a000 2026-02-11T23:31:17.615+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.615+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.615+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 712 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8d384ce8:::obj297:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3e4a000 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.615+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8d384ce8:::obj297:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.615+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8d384ce8:::obj297:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8d384ce8:::obj297:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8d6bd12e:::obj368:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8d6bd12e:::obj368:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8d6bd12e:::obj368:head 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8d6bd12e:::obj368:head v 107'368 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.615+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.615+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8d6bd12e:::obj368:head, version: 107'368, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8d6bd12e:::obj368:head@107'368, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8d384ce8:::obj297:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8d384ce8:::obj297:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 115/115 lb 4:8d384ce8:::obj297:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8d384ce8:::obj297:head) v3 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d29890f:::obj305:head local-lis/les=112/113 n=331 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8d384ce8:::obj297:head) 2026-02-11T23:31:17.615+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8d384ce8:::obj297:head) v3 finish latency 0.000011 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.615+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.615+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.615+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0780 con 0x3e4a000 2026-02-11T23:31:17.615+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8d6bd12e:::obj368:head)]) -- 0x4ae4180 con 0x3e4a000 2026-02-11T23:31:17.615+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.616+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 713 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x43ea000 con 0x3e4a000 2026-02-11T23:31:17.616+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.616+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.616+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 714 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8d6bd12e:::obj368:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3e4a000 2026-02-11T23:31:17.616+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8d6bd12e:::obj368:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.616+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8d6bd12e:::obj368:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8d6bd12e:::obj368:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8dd16f86:::obj3:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8dd16f86:::obj3:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8dd16f86:::obj3:head 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8dd16f86:::obj3:head v 107'3 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.616+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8dd16f86:::obj3:head, version: 107'3, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8dd16f86:::obj3:head@107'3, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8d6bd12e:::obj368:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.616+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8d6bd12e:::obj368:head) v3 prio 15 cost 0 latency 0.000058 pg_backfill(progress 4.0 e 115/115 lb 4:8d6bd12e:::obj368:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.616+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.616+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8d6bd12e:::obj368:head) v3 2026-02-11T23:31:17.616+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d384ce8:::obj297:head local-lis/les=112/113 n=332 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8d6bd12e:::obj368:head) 2026-02-11T23:31:17.616+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.616+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8d6bd12e:::obj368:head) v3 finish latency 0.000010 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.616+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.616+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0900 con 0x3e4a000 2026-02-11T23:31:17.616+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8dd16f86:::obj3:head)]) -- 0x4ae4f00 con 0x3e4a000 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.616+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 715 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51821a0 con 0x3e4a000 2026-02-11T23:31:17.616+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.616+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.616+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 716 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8dd16f86:::obj3:head) ==== 1060+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3e4a000 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.616+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8dd16f86:::obj3:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.616+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8dd16f86:::obj3:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.616+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8dd16f86:::obj3:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8e988874:::obj200:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.616+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8e988874:::obj200:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.616+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8e988874:::obj200:head 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8e988874:::obj200:head v 107'200 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.617+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8e988874:::obj200:head, version: 107'200, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8e988874:::obj200:head@107'200, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.617+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8dd16f86:::obj3:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8dd16f86:::obj3:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:8dd16f86:::obj3:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8dd16f86:::obj3:head) v3 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8d6bd12e:::obj368:head local-lis/les=112/113 n=333 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8dd16f86:::obj3:head) 2026-02-11T23:31:17.617+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8dd16f86:::obj3:head) v3 finish latency 0.000011 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.617+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.617+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0a80 con 0x3e4a000 2026-02-11T23:31:17.617+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8e988874:::obj200:head)]) -- 0x4c1f080 con 0x3e4a000 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.617+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 717 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3e4a000 2026-02-11T23:31:17.617+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.617+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.617+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.617+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 718 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8e988874:::obj200:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3e4a000 2026-02-11T23:31:17.617+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.617+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8e988874:::obj200:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.617+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8e988874:::obj200:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.617+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.617+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8e988874:::obj200:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8eb14ea4:::obj384:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.617+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8eb14ea4:::obj384:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8eb14ea4:::obj384:head 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8eb14ea4:::obj384:head v 107'384 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.617+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.617+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8eb14ea4:::obj384:head, version: 107'384, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8eb14ea4:::obj384:head@107'384, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.617+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8e988874:::obj200:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8e988874:::obj200:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 115/115 lb 4:8e988874:::obj200:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8e988874:::obj200:head) v3 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8dd16f86:::obj3:head local-lis/les=112/113 n=334 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8e988874:::obj200:head) 2026-02-11T23:31:17.617+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8e988874:::obj200:head) v3 finish latency 0.000010 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.617+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.617+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.617+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0c00 con 0x3e4a000 2026-02-11T23:31:17.617+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8eb14ea4:::obj384:head)]) -- 0x511bc80 con 0x3e4a000 2026-02-11T23:31:17.617+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.618+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 719 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51831e0 con 0x3e4a000 2026-02-11T23:31:17.618+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.618+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.618+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 720 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8eb14ea4:::obj384:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3e4a000 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.618+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8eb14ea4:::obj384:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.618+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8eb14ea4:::obj384:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8eb14ea4:::obj384:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8f4c4fa9:::obj590:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8f4c4fa9:::obj590:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8f4c4fa9:::obj590:head 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8f4c4fa9:::obj590:head v 107'590 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.618+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8f4c4fa9:::obj590:head, version: 107'590, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f4c4fa9:::obj590:head@107'590, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8eb14ea4:::obj384:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.618+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8eb14ea4:::obj384:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:8eb14ea4:::obj384:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.618+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.618+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8eb14ea4:::obj384:head) v3 2026-02-11T23:31:17.618+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8e988874:::obj200:head local-lis/les=112/113 n=335 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8eb14ea4:::obj384:head) 2026-02-11T23:31:17.618+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.618+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8eb14ea4:::obj384:head) v3 finish latency 0.000011 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.618+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.618+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0d80 con 0x3e4a000 2026-02-11T23:31:17.618+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8f4c4fa9:::obj590:head)]) -- 0x4e5d680 con 0x3e4a000 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.618+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 721 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3e4a000 2026-02-11T23:31:17.618+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.618+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.618+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 722 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8f4c4fa9:::obj590:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3e4a000 2026-02-11T23:31:17.618+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8f4c4fa9:::obj590:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.618+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8f4c4fa9:::obj590:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8f4c4fa9:::obj590:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8f7cdb2a:::obj538:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8f7cdb2a:::obj538:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8f7cdb2a:::obj538:head 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8f7cdb2a:::obj538:head v 107'538 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.618+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.618+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8f7cdb2a:::obj538:head, version: 107'538, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8f7cdb2a:::obj538:head@107'538, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.618+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8f4c4fa9:::obj590:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.618+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8f4c4fa9:::obj590:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 115/115 lb 4:8f4c4fa9:::obj590:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.618+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.618+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8f4c4fa9:::obj590:head) v3 2026-02-11T23:31:17.618+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8eb14ea4:::obj384:head local-lis/les=112/113 n=336 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8f4c4fa9:::obj590:head) 2026-02-11T23:31:17.618+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.618+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8f4c4fa9:::obj590:head) v3 finish latency 0.000011 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.618+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.619+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.619+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0f00 con 0x3e4a000 2026-02-11T23:31:17.619+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8f7cdb2a:::obj538:head)]) -- 0x4ae5500 con 0x3e4a000 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.619+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 723 ==== MOSDPGPush(4.0 115/112 [PushOp(4:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.619+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.619+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.619+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 724 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8f7cdb2a:::obj538:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3e4a000 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.619+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8f7cdb2a:::obj538:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.619+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8f7cdb2a:::obj538:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8f7cdb2a:::obj538:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:8fcfcc34:::obj221:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:8fcfcc34:::obj221:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:8fcfcc34:::obj221:head 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:8fcfcc34:::obj221:head v 107'221 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.619+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:8fcfcc34:::obj221:head, version: 107'221, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:8fcfcc34:::obj221:head@107'221, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8f7cdb2a:::obj538:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.619+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8f7cdb2a:::obj538:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:8f7cdb2a:::obj538:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.619+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.619+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8f7cdb2a:::obj538:head) v3 2026-02-11T23:31:17.619+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f4c4fa9:::obj590:head local-lis/les=112/113 n=337 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8f7cdb2a:::obj538:head) 2026-02-11T23:31:17.619+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.619+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8f7cdb2a:::obj538:head) v3 finish latency 0.000010 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.619+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.619+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1080 con 0x3e4a000 2026-02-11T23:31:17.619+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:8fcfcc34:::obj221:head)]) -- 0x49d7800 con 0x3e4a000 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.619+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 725 ==== MOSDPGPush(4.0 115/112 [PushOp(4:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3a00 con 0x3e4a000 2026-02-11T23:31:17.619+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.619+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.619+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 726 ==== pg_backfill(progress 4.0 e 115/115 lb 4:8fcfcc34:::obj221:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3e4a000 2026-02-11T23:31:17.619+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:8fcfcc34:::obj221:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.619+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8fcfcc34:::obj221:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.619+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8fcfcc34:::obj221:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:902252c4:::obj397:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.619+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:902252c4:::obj397:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:902252c4:::obj397:head 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:902252c4:::obj397:head v 107'397 2026-02-11T23:31:17.619+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.619+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.620+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:902252c4:::obj397:head, version: 107'397, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:902252c4:::obj397:head@107'397, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.620+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:8fcfcc34:::obj221:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8fcfcc34:::obj221:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 115/115 lb 4:8fcfcc34:::obj221:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:8fcfcc34:::obj221:head) v3 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8f7cdb2a:::obj538:head local-lis/les=112/113 n=338 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:8fcfcc34:::obj221:head) 2026-02-11T23:31:17.620+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:8fcfcc34:::obj221:head) v3 finish latency 0.000010 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.620+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.620+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1200 con 0x3e4a000 2026-02-11T23:31:17.620+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:902252c4:::obj397:head)]) -- 0x511bb00 con 0x3e4a000 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.620+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 727 ==== MOSDPGPush(4.0 115/112 [PushOp(4:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.620+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.620+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.620+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.620+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.620+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.620+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 728 ==== pg_backfill(progress 4.0 e 115/115 lb 4:902252c4:::obj397:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded080 con 0x3e4a000 2026-02-11T23:31:17.620+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:902252c4:::obj397:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.620+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.620+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:902252c4:::obj397:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:902252c4:::obj397:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.620+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.620+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.620+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.620+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.620+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:90d5bfbc:::obj593:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.620+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:90d5bfbc:::obj593:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.620+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:90d5bfbc:::obj593:head 2026-02-11T23:31:17.620+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:90d5bfbc:::obj593:head v 107'593 2026-02-11T23:31:17.620+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.620+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.620+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:90d5bfbc:::obj593:head, version: 107'593, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:90d5bfbc:::obj593:head@107'593, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.620+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:902252c4:::obj397:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:902252c4:::obj397:head) v3 prio 15 cost 0 latency 0.000056 pg_backfill(progress 4.0 e 115/115 lb 4:902252c4:::obj397:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:902252c4:::obj397:head) v3 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:8fcfcc34:::obj221:head local-lis/les=112/113 n=339 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:902252c4:::obj397:head) 2026-02-11T23:31:17.620+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:902252c4:::obj397:head) v3 finish latency 0.000011 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.620+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.620+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.620+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1380 con 0x3e4a000 2026-02-11T23:31:17.620+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:90d5bfbc:::obj593:head)]) -- 0x4c1f380 con 0x3e4a000 2026-02-11T23:31:17.620+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.621+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 729 ==== MOSDPGPush(4.0 115/112 [PushOp(4:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.621+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.621+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.621+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 730 ==== pg_backfill(progress 4.0 e 115/115 lb 4:90d5bfbc:::obj593:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3e4a000 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.621+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:90d5bfbc:::obj593:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.621+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:90d5bfbc:::obj593:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:90d5bfbc:::obj593:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:917e3523:::obj419:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:917e3523:::obj419:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:917e3523:::obj419:head 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:917e3523:::obj419:head v 107'419 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.621+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:917e3523:::obj419:head, version: 107'419, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:917e3523:::obj419:head@107'419, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:90d5bfbc:::obj593:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:90d5bfbc:::obj593:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 115/115 lb 4:90d5bfbc:::obj593:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:90d5bfbc:::obj593:head) v3 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:902252c4:::obj397:head local-lis/les=112/113 n=340 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:90d5bfbc:::obj593:head) 2026-02-11T23:31:17.621+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:90d5bfbc:::obj593:head) v3 finish latency 0.000010 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.621+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1500 con 0x3e4a000 2026-02-11T23:31:17.621+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:917e3523:::obj419:head)]) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.621+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 731 ==== MOSDPGPush(4.0 115/112 [PushOp(4:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3e4a000 2026-02-11T23:31:17.621+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.621+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.621+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 732 ==== pg_backfill(progress 4.0 e 115/115 lb 4:917e3523:::obj419:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3e4a000 2026-02-11T23:31:17.621+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:917e3523:::obj419:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.621+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:917e3523:::obj419:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:917e3523:::obj419:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:91fc975b:::obj401:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:91fc975b:::obj401:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:91fc975b:::obj401:head 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:91fc975b:::obj401:head v 107'401 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.621+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.621+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:91fc975b:::obj401:head, version: 107'401, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:91fc975b:::obj401:head@107'401, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:917e3523:::obj419:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:917e3523:::obj419:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 115/115 lb 4:917e3523:::obj419:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:917e3523:::obj419:head) v3 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:90d5bfbc:::obj593:head local-lis/les=112/113 n=341 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:917e3523:::obj419:head) 2026-02-11T23:31:17.621+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:917e3523:::obj419:head) v3 finish latency 0.000010 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.621+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.621+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.621+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1680 con 0x3e4a000 2026-02-11T23:31:17.621+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:91fc975b:::obj401:head)]) -- 0x4c2de00 con 0x3e4a000 2026-02-11T23:31:17.621+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.622+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 733 ==== MOSDPGPush(4.0 115/112 [PushOp(4:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183d40 con 0x3e4a000 2026-02-11T23:31:17.622+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.622+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.622+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 734 ==== pg_backfill(progress 4.0 e 115/115 lb 4:91fc975b:::obj401:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3e4a000 2026-02-11T23:31:17.622+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:91fc975b:::obj401:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.622+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:91fc975b:::obj401:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:91fc975b:::obj401:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:92b89151:::obj431:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:92b89151:::obj431:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:92b89151:::obj431:head 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:92b89151:::obj431:head v 107'431 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.622+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:92b89151:::obj431:head, version: 107'431, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:92b89151:::obj431:head@107'431, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:91fc975b:::obj401:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.622+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:91fc975b:::obj401:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:91fc975b:::obj401:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.622+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.622+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:91fc975b:::obj401:head) v3 2026-02-11T23:31:17.622+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:917e3523:::obj419:head local-lis/les=112/113 n=342 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:91fc975b:::obj401:head) 2026-02-11T23:31:17.622+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.622+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:91fc975b:::obj401:head) v3 finish latency 0.000010 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.622+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.622+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1800 con 0x3e4a000 2026-02-11T23:31:17.622+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:92b89151:::obj431:head)]) -- 0x4c1fc80 con 0x3e4a000 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.622+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 735 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3520 con 0x3e4a000 2026-02-11T23:31:17.622+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.622+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.622+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 736 ==== pg_backfill(progress 4.0 e 115/115 lb 4:92b89151:::obj431:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3e4a000 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.622+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:92b89151:::obj431:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 115 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.622+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:92b89151:::obj431:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:92b89151:::obj431:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9315b0b3:::obj555:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9315b0b3:::obj555:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9315b0b3:::obj555:head 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9315b0b3:::obj555:head v 107'555 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.622+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.622+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9315b0b3:::obj555:head, version: 107'555, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9315b0b3:::obj555:head@107'555, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.622+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:92b89151:::obj431:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.622+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:92b89151:::obj431:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 4.0 e 115/115 lb 4:92b89151:::obj431:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.622+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.622+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:92b89151:::obj431:head) v3 2026-02-11T23:31:17.622+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:91fc975b:::obj401:head local-lis/les=112/113 n=343 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:92b89151:::obj431:head) 2026-02-11T23:31:17.622+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.622+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:92b89151:::obj431:head) v3 finish latency 0.000011 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.622+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.623+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.623+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.623+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1980 con 0x3e4a000 2026-02-11T23:31:17.623+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9315b0b3:::obj555:head)]) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.623+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 737 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dfe1a0 con 0x3e4a000 2026-02-11T23:31:17.623+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.623+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.623+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.623+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 738 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9315b0b3:::obj555:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd180 con 0x3e4a000 2026-02-11T23:31:17.623+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.623+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9315b0b3:::obj555:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.623+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9315b0b3:::obj555:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.623+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.623+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.623+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.623+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9315b0b3:::obj555:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.623+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.623+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.623+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9397791a:::obj547:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.623+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9397791a:::obj547:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.623+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9397791a:::obj547:head 2026-02-11T23:31:17.623+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9397791a:::obj547:head v 107'547 2026-02-11T23:31:17.623+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.623+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.623+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9397791a:::obj547:head, version: 107'547, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9397791a:::obj547:head@107'547, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.623+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9315b0b3:::obj555:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.623+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9315b0b3:::obj555:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 115/115 lb 4:9315b0b3:::obj555:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.623+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.623+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9315b0b3:::obj555:head) v3 2026-02-11T23:31:17.623+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:92b89151:::obj431:head local-lis/les=112/113 n=344 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9315b0b3:::obj555:head) 2026-02-11T23:31:17.623+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.623+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9315b0b3:::obj555:head) v3 finish latency 0.000010 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.623+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.623+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.623+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1b00 con 0x3e4a000 2026-02-11T23:31:17.623+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9397791a:::obj547:head)]) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.623+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.624+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 739 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2b60 con 0x3e4a000 2026-02-11T23:31:17.624+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.624+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.624+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 740 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9397791a:::obj547:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def180 con 0x3e4a000 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.624+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9397791a:::obj547:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.624+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9397791a:::obj547:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9397791a:::obj547:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9455f78c:::obj228:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9455f78c:::obj228:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9455f78c:::obj228:head 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9455f78c:::obj228:head v 107'228 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.624+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9455f78c:::obj228:head, version: 107'228, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9455f78c:::obj228:head@107'228, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9397791a:::obj547:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9397791a:::obj547:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:9397791a:::obj547:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9397791a:::obj547:head) v3 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9315b0b3:::obj555:head local-lis/les=112/113 n=345 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9397791a:::obj547:head) 2026-02-11T23:31:17.624+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9397791a:::obj547:head) v3 finish latency 0.000009 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.624+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1c80 con 0x3e4a000 2026-02-11T23:31:17.624+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9455f78c:::obj228:head)]) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.624+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 741 ==== MOSDPGPush(4.0 115/112 [PushOp(4:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2680 con 0x3e4a000 2026-02-11T23:31:17.624+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.624+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.624+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 742 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9455f78c:::obj228:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3e4a000 2026-02-11T23:31:17.624+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9455f78c:::obj228:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.624+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9455f78c:::obj228:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9455f78c:::obj228:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:945b6e2e:::obj114:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:945b6e2e:::obj114:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:945b6e2e:::obj114:head 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:945b6e2e:::obj114:head v 107'114 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.624+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.624+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:945b6e2e:::obj114:head, version: 107'114, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:945b6e2e:::obj114:head@107'114, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9455f78c:::obj228:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9455f78c:::obj228:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 115/115 lb 4:9455f78c:::obj228:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9455f78c:::obj228:head) v3 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9397791a:::obj547:head local-lis/les=112/113 n=346 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9455f78c:::obj228:head) 2026-02-11T23:31:17.624+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9455f78c:::obj228:head) v3 finish latency 0.000010 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.624+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.624+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.624+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1e00 con 0x3e4a000 2026-02-11T23:31:17.624+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:945b6e2e:::obj114:head)]) -- 0x4c1e300 con 0x3e4a000 2026-02-11T23:31:17.624+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.625+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 743 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a21a0 con 0x3e4a000 2026-02-11T23:31:17.625+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.625+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.625+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 744 ==== pg_backfill(progress 4.0 e 115/115 lb 4:945b6e2e:::obj114:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164c00 con 0x3e4a000 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.625+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:945b6e2e:::obj114:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.625+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:945b6e2e:::obj114:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:945b6e2e:::obj114:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9477fafa:::obj499:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9477fafa:::obj499:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9477fafa:::obj499:head 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9477fafa:::obj499:head v 107'499 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.625+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9477fafa:::obj499:head, version: 107'499, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9477fafa:::obj499:head@107'499, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:945b6e2e:::obj114:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.625+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:945b6e2e:::obj114:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 4.0 e 115/115 lb 4:945b6e2e:::obj114:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.625+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.625+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:945b6e2e:::obj114:head) v3 2026-02-11T23:31:17.625+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9455f78c:::obj228:head local-lis/les=112/113 n=347 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:945b6e2e:::obj114:head) 2026-02-11T23:31:17.625+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.625+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:945b6e2e:::obj114:head) v3 finish latency 0.000010 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.625+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.625+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2000 con 0x3e4a000 2026-02-11T23:31:17.625+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9477fafa:::obj499:head)]) -- 0x4ba2d80 con 0x3e4a000 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.625+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 745 ==== MOSDPGPush(4.0 115/112 [PushOp(4:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.625+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.625+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.625+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 746 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9477fafa:::obj499:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165180 con 0x3e4a000 2026-02-11T23:31:17.625+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9477fafa:::obj499:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.625+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9477fafa:::obj499:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9477fafa:::obj499:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:95a9a074:::obj509:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:95a9a074:::obj509:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:95a9a074:::obj509:head 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:95a9a074:::obj509:head v 107'509 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.625+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.625+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:95a9a074:::obj509:head, version: 107'509, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:95a9a074:::obj509:head@107'509, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.625+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9477fafa:::obj499:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.625+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9477fafa:::obj499:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 4.0 e 115/115 lb 4:9477fafa:::obj499:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.625+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.625+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9477fafa:::obj499:head) v3 2026-02-11T23:31:17.625+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:945b6e2e:::obj114:head local-lis/les=112/113 n=348 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9477fafa:::obj499:head) 2026-02-11T23:31:17.625+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.625+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9477fafa:::obj499:head) v3 finish latency 0.000010 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.625+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.626+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.626+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.626+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2180 con 0x3e4a000 2026-02-11T23:31:17.626+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:95a9a074:::obj509:head)]) -- 0x4be9e00 con 0x3e4a000 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.626+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 747 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.626+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.626+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.626+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.626+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 748 ==== pg_backfill(progress 4.0 e 115/115 lb 4:95a9a074:::obj509:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3e4a000 2026-02-11T23:31:17.626+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.626+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:95a9a074:::obj509:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.626+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:95a9a074:::obj509:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.626+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.626+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.626+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.626+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:95a9a074:::obj509:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.626+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.626+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.626+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9604ab91:::obj494:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.626+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9604ab91:::obj494:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.626+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9604ab91:::obj494:head 2026-02-11T23:31:17.626+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9604ab91:::obj494:head v 107'494 2026-02-11T23:31:17.626+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.626+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.626+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9604ab91:::obj494:head, version: 107'494, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9604ab91:::obj494:head@107'494, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:17.626+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:95a9a074:::obj509:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.626+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:95a9a074:::obj509:head) v3 prio 15 cost 0 latency 0.000058 pg_backfill(progress 4.0 e 115/115 lb 4:95a9a074:::obj509:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.626+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.626+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:95a9a074:::obj509:head) v3 2026-02-11T23:31:17.626+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9477fafa:::obj499:head local-lis/les=112/113 n=349 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:95a9a074:::obj509:head) 2026-02-11T23:31:17.626+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.626+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:95a9a074:::obj509:head) v3 finish latency 0.000010 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.626+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.626+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.626+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2300 con 0x3e4a000 2026-02-11T23:31:17.626+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9604ab91:::obj494:head)]) -- 0x4be9b00 con 0x3e4a000 2026-02-11T23:31:17.626+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.627+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 749 ==== MOSDPGPush(4.0 115/112 [PushOp(4:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.627+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.627+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.627+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 750 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9604ab91:::obj494:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.627+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9604ab91:::obj494:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.627+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9604ab91:::obj494:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9604ab91:::obj494:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:96183f21:::obj585:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:96183f21:::obj585:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:96183f21:::obj585:head 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:96183f21:::obj585:head v 107'585 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.627+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:96183f21:::obj585:head, version: 107'585, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96183f21:::obj585:head@107'585, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9604ab91:::obj494:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9604ab91:::obj494:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 115/115 lb 4:9604ab91:::obj494:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9604ab91:::obj494:head) v3 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:95a9a074:::obj509:head local-lis/les=112/113 n=350 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9604ab91:::obj494:head) 2026-02-11T23:31:17.627+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9604ab91:::obj494:head) v3 finish latency 0.000018 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.627+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2480 con 0x3e4a000 2026-02-11T23:31:17.627+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:96183f21:::obj585:head)]) -- 0x4be9800 con 0x3e4a000 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.627+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 751 ==== MOSDPGPush(4.0 115/112 [PushOp(4:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.627+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.627+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.627+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 752 ==== pg_backfill(progress 4.0 e 115/115 lb 4:96183f21:::obj585:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.627+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:96183f21:::obj585:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.627+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:96183f21:::obj585:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:96183f21:::obj585:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:96d0cad0:::obj307:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:96d0cad0:::obj307:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:96d0cad0:::obj307:head 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:96d0cad0:::obj307:head v 107'307 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.627+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.627+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:96d0cad0:::obj307:head, version: 107'307, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:96d0cad0:::obj307:head@107'307, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:96183f21:::obj585:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:96183f21:::obj585:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 115/115 lb 4:96183f21:::obj585:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:96183f21:::obj585:head) v3 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9604ab91:::obj494:head local-lis/les=112/113 n=351 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:96183f21:::obj585:head) 2026-02-11T23:31:17.627+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:96183f21:::obj585:head) v3 finish latency 0.000016 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.627+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.627+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.627+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2600 con 0x3e4a000 2026-02-11T23:31:17.627+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:96d0cad0:::obj307:head)]) -- 0x4be9500 con 0x3e4a000 2026-02-11T23:31:17.627+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.628+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 753 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183040 con 0x3e4a000 2026-02-11T23:31:17.628+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.628+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.628+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 754 ==== pg_backfill(progress 4.0 e 115/115 lb 4:96d0cad0:::obj307:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.628+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:96d0cad0:::obj307:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.628+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:96d0cad0:::obj307:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:96d0cad0:::obj307:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9720f72d:::obj90:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9720f72d:::obj90:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9720f72d:::obj90:head 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9720f72d:::obj90:head v 107'90 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.628+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9720f72d:::obj90:head, version: 107'90, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9720f72d:::obj90:head@107'90, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:96d0cad0:::obj307:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.628+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:96d0cad0:::obj307:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:96d0cad0:::obj307:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.628+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.628+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:96d0cad0:::obj307:head) v3 2026-02-11T23:31:17.628+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96183f21:::obj585:head local-lis/les=112/113 n=352 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:96d0cad0:::obj307:head) 2026-02-11T23:31:17.628+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.628+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:96d0cad0:::obj307:head) v3 finish latency 0.000017 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.628+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.628+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2780 con 0x3e4a000 2026-02-11T23:31:17.628+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9720f72d:::obj90:head)]) -- 0x4be9200 con 0x3e4a000 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.628+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 755 ==== MOSDPGPush(4.0 115/112 [PushOp(4:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4680 con 0x3e4a000 2026-02-11T23:31:17.628+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.628+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.628+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 756 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9720f72d:::obj90:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.628+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9720f72d:::obj90:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.628+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9720f72d:::obj90:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9720f72d:::obj90:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:98019cfb:::obj381:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:98019cfb:::obj381:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:98019cfb:::obj381:head 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:98019cfb:::obj381:head v 107'381 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.628+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.628+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:98019cfb:::obj381:head, version: 107'381, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98019cfb:::obj381:head@107'381, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:17.628+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9720f72d:::obj90:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.628+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9720f72d:::obj90:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 4.0 e 115/115 lb 4:9720f72d:::obj90:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.628+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.628+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9720f72d:::obj90:head) v3 2026-02-11T23:31:17.628+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:96d0cad0:::obj307:head local-lis/les=112/113 n=353 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9720f72d:::obj90:head) 2026-02-11T23:31:17.628+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.628+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9720f72d:::obj90:head) v3 finish latency 0.000016 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.628+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.629+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.629+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.629+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2900 con 0x3e4a000 2026-02-11T23:31:17.629+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:98019cfb:::obj381:head)]) -- 0x4ae4480 con 0x3e4a000 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.629+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 757 ==== MOSDPGPush(4.0 115/112 [PushOp(4:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516da00 con 0x3e4a000 2026-02-11T23:31:17.629+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.629+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.629+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.629+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.629+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 758 ==== pg_backfill(progress 4.0 e 115/115 lb 4:98019cfb:::obj381:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.629+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:98019cfb:::obj381:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.629+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.629+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:98019cfb:::obj381:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.629+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.629+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.629+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.629+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.629+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:98019cfb:::obj381:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.629+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:98b68518:::obj150:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.629+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:98b68518:::obj150:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.629+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:98b68518:::obj150:head 2026-02-11T23:31:17.629+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:98b68518:::obj150:head v 107'150 2026-02-11T23:31:17.629+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.629+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.629+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:98b68518:::obj150:head, version: 107'150, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:98b68518:::obj150:head@107'150, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.629+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:98019cfb:::obj381:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.629+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:98019cfb:::obj381:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 4.0 e 115/115 lb 4:98019cfb:::obj381:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.629+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.629+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:98019cfb:::obj381:head) v3 2026-02-11T23:31:17.629+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9720f72d:::obj90:head local-lis/les=112/113 n=354 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:98019cfb:::obj381:head) 2026-02-11T23:31:17.629+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.629+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:98019cfb:::obj381:head) v3 finish latency 0.000015 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.629+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.629+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.629+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2a80 con 0x3e4a000 2026-02-11T23:31:17.629+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:98b68518:::obj150:head)]) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.629+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.630+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 759 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182000 con 0x3e4a000 2026-02-11T23:31:17.630+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 115 2026-02-11T23:31:17.630+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.630+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 760 ==== pg_backfill(progress 4.0 e 115/115 lb 4:98b68518:::obj150:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.630+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:98b68518:::obj150:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.630+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:98b68518:::obj150:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:98b68518:::obj150:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9956259d:::obj240:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9956259d:::obj240:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9956259d:::obj240:head 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9956259d:::obj240:head v 107'240 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.630+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9956259d:::obj240:head, version: 107'240, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9956259d:::obj240:head@107'240, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:98b68518:::obj150:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:98b68518:::obj150:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 115/115 lb 4:98b68518:::obj150:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:98b68518:::obj150:head) v3 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98019cfb:::obj381:head local-lis/les=112/113 n=355 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:98b68518:::obj150:head) 2026-02-11T23:31:17.630+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:98b68518:::obj150:head) v3 finish latency 0.000015 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.630+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2c00 con 0x3e4a000 2026-02-11T23:31:17.630+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9956259d:::obj240:head)]) -- 0x4ae4600 con 0x3e4a000 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.630+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 761 ==== MOSDPGPush(4.0 115/112 [PushOp(4:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51824e0 con 0x3e4a000 2026-02-11T23:31:17.630+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.630+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.630+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 762 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9956259d:::obj240:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.630+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9956259d:::obj240:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.630+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9956259d:::obj240:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9956259d:::obj240:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:996db15f:::obj326:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:996db15f:::obj326:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:996db15f:::obj326:head 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:996db15f:::obj326:head v 107'326 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.630+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.630+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:996db15f:::obj326:head, version: 107'326, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:996db15f:::obj326:head@107'326, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9956259d:::obj240:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9956259d:::obj240:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 115/115 lb 4:9956259d:::obj240:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9956259d:::obj240:head) v3 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:98b68518:::obj150:head local-lis/les=112/113 n=356 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9956259d:::obj240:head) 2026-02-11T23:31:17.630+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9956259d:::obj240:head) v3 finish latency 0.000015 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.630+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.630+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.630+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2d80 con 0x3e4a000 2026-02-11T23:31:17.630+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:996db15f:::obj326:head)]) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.630+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.631+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 763 ==== MOSDPGPush(4.0 115/112 [PushOp(4:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3e4a000 2026-02-11T23:31:17.631+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.631+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.631+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 764 ==== pg_backfill(progress 4.0 e 115/115 lb 4:996db15f:::obj326:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163b80 con 0x3e4a000 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.631+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:996db15f:::obj326:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.631+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:996db15f:::obj326:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:996db15f:::obj326:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:99944c24:::obj170:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:99944c24:::obj170:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:99944c24:::obj170:head 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:99944c24:::obj170:head v 107'170 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.631+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:99944c24:::obj170:head, version: 107'170, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:99944c24:::obj170:head@107'170, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:996db15f:::obj326:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.631+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:996db15f:::obj326:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:996db15f:::obj326:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.631+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.631+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:996db15f:::obj326:head) v3 2026-02-11T23:31:17.631+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9956259d:::obj240:head local-lis/les=112/113 n=357 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:996db15f:::obj326:head) 2026-02-11T23:31:17.631+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.631+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:996db15f:::obj326:head) v3 finish latency 0.000015 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.631+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.631+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2f00 con 0x3e4a000 2026-02-11T23:31:17.631+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:99944c24:::obj170:head)]) -- 0x4ae4780 con 0x3e4a000 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.631+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 765 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x43f2680 con 0x3e4a000 2026-02-11T23:31:17.631+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.631+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.631+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 766 ==== pg_backfill(progress 4.0 e 115/115 lb 4:99944c24:::obj170:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163600 con 0x3e4a000 2026-02-11T23:31:17.631+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:99944c24:::obj170:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.631+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:99944c24:::obj170:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:99944c24:::obj170:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9a09113a:::obj7:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9a09113a:::obj7:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9a09113a:::obj7:head 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9a09113a:::obj7:head v 107'7 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.631+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.631+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9a09113a:::obj7:head, version: 107'7, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9a09113a:::obj7:head@107'7, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:17.631+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:99944c24:::obj170:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.631+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:99944c24:::obj170:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 4.0 e 115/115 lb 4:99944c24:::obj170:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.631+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.631+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:99944c24:::obj170:head) v3 2026-02-11T23:31:17.631+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:996db15f:::obj326:head local-lis/les=112/113 n=358 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:99944c24:::obj170:head) 2026-02-11T23:31:17.631+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.631+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:99944c24:::obj170:head) v3 finish latency 0.000014 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.631+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.632+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.632+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.632+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3080 con 0x3e4a000 2026-02-11T23:31:17.632+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9a09113a:::obj7:head)]) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.632+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 767 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.632+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.632+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.632+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.632+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 768 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9a09113a:::obj7:head) ==== 1060+0+0 (unknown 0 0 0) 0x5163080 con 0x3e4a000 2026-02-11T23:31:17.632+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9a09113a:::obj7:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.632+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.632+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9a09113a:::obj7:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.632+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.632+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9a09113a:::obj7:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.632+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.632+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.632+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.632+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.632+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9bbc8a19:::obj348:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.632+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9bbc8a19:::obj348:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.632+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9bbc8a19:::obj348:head 2026-02-11T23:31:17.632+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9bbc8a19:::obj348:head v 107'348 2026-02-11T23:31:17.632+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.632+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.632+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9bbc8a19:::obj348:head, version: 107'348, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9bbc8a19:::obj348:head@107'348, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.632+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9a09113a:::obj7:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.632+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9a09113a:::obj7:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 115/115 lb 4:9a09113a:::obj7:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.632+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.632+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9a09113a:::obj7:head) v3 2026-02-11T23:31:17.632+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:99944c24:::obj170:head local-lis/les=112/113 n=359 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9a09113a:::obj7:head) 2026-02-11T23:31:17.632+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.632+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9a09113a:::obj7:head) v3 finish latency 0.000013 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.632+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.632+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.632+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3200 con 0x3e4a000 2026-02-11T23:31:17.632+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9bbc8a19:::obj348:head)]) -- 0x4be8180 con 0x3e4a000 2026-02-11T23:31:17.632+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.632+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 769 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.632+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.632+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.632+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.632+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.632+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 770 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9bbc8a19:::obj348:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3e4a000 2026-02-11T23:31:17.633+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9bbc8a19:::obj348:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 115 2026-02-11T23:31:17.633+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9bbc8a19:::obj348:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9bbc8a19:::obj348:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9c3090f1:::obj522:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9c3090f1:::obj522:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9c3090f1:::obj522:head 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9c3090f1:::obj522:head v 107'522 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.633+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9c3090f1:::obj522:head, version: 107'522, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c3090f1:::obj522:head@107'522, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9bbc8a19:::obj348:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9bbc8a19:::obj348:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 115/115 lb 4:9bbc8a19:::obj348:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9bbc8a19:::obj348:head) v3 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9a09113a:::obj7:head local-lis/les=112/113 n=360 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9bbc8a19:::obj348:head) 2026-02-11T23:31:17.633+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9bbc8a19:::obj348:head) v3 finish latency 0.000015 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.633+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3380 con 0x3e4a000 2026-02-11T23:31:17.633+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9c3090f1:::obj522:head)]) -- 0x4f50180 con 0x3e4a000 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.633+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 771 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.633+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.633+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.633+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 772 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9c3090f1:::obj522:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162580 con 0x3e4a000 2026-02-11T23:31:17.633+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9c3090f1:::obj522:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.633+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9c3090f1:::obj522:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9c3090f1:::obj522:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9c626db4:::obj119:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9c626db4:::obj119:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9c626db4:::obj119:head 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9c626db4:::obj119:head v 107'119 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.633+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.633+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9c626db4:::obj119:head, version: 107'119, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c626db4:::obj119:head@107'119, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9c3090f1:::obj522:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9c3090f1:::obj522:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 115/115 lb 4:9c3090f1:::obj522:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9c3090f1:::obj522:head) v3 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9bbc8a19:::obj348:head local-lis/les=112/113 n=361 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9c3090f1:::obj522:head) 2026-02-11T23:31:17.633+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9c3090f1:::obj522:head) v3 finish latency 0.000014 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.633+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.633+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.633+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3500 con 0x3e4a000 2026-02-11T23:31:17.633+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9c626db4:::obj119:head)]) -- 0x4c1e300 con 0x3e4a000 2026-02-11T23:31:17.633+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.634+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 773 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.634+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.634+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.634+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 774 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9c626db4:::obj119:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162000 con 0x3e4a000 2026-02-11T23:31:17.634+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9c626db4:::obj119:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.634+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9c626db4:::obj119:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9c626db4:::obj119:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9c9438f5:::obj513:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9c9438f5:::obj513:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9c9438f5:::obj513:head 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9c9438f5:::obj513:head v 107'513 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.634+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9c9438f5:::obj513:head, version: 107'513, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9c9438f5:::obj513:head@107'513, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000044 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9c626db4:::obj119:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.634+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9c626db4:::obj119:head) v3 prio 15 cost 0 latency 0.000053 pg_backfill(progress 4.0 e 115/115 lb 4:9c626db4:::obj119:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.634+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.634+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9c626db4:::obj119:head) v3 2026-02-11T23:31:17.634+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c3090f1:::obj522:head local-lis/les=112/113 n=362 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9c626db4:::obj119:head) 2026-02-11T23:31:17.634+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.634+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9c626db4:::obj119:head) v3 finish latency 0.000013 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.634+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.634+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3680 con 0x3e4a000 2026-02-11T23:31:17.634+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9c9438f5:::obj513:head)]) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.634+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 775 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2340 con 0x3e4a000 2026-02-11T23:31:17.634+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.634+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.634+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 776 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9c9438f5:::obj513:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3e4a000 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.634+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9c9438f5:::obj513:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.634+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9c9438f5:::obj513:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9c9438f5:::obj513:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9f41ce81:::obj424:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9f41ce81:::obj424:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9f41ce81:::obj424:head 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9f41ce81:::obj424:head v 107'424 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.634+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.634+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9f41ce81:::obj424:head, version: 107'424, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f41ce81:::obj424:head@107'424, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.634+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9c9438f5:::obj513:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.634+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9c9438f5:::obj513:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:9c9438f5:::obj513:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.634+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.634+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9c9438f5:::obj513:head) v3 2026-02-11T23:31:17.634+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c626db4:::obj119:head local-lis/les=112/113 n=363 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9c9438f5:::obj513:head) 2026-02-11T23:31:17.634+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.634+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9c9438f5:::obj513:head) v3 finish latency 0.000013 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.634+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.635+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.635+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.635+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3800 con 0x3e4a000 2026-02-11T23:31:17.635+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9f41ce81:::obj424:head)]) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.635+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 777 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3e4a000 2026-02-11T23:31:17.635+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.635+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.635+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.635+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 778 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9f41ce81:::obj424:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3e4a000 2026-02-11T23:31:17.635+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.635+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9f41ce81:::obj424:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.635+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9f41ce81:::obj424:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.635+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.635+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.635+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.635+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9f41ce81:::obj424:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.635+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.635+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.635+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9f7e105e:::obj535:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.635+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9f7e105e:::obj535:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.635+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9f7e105e:::obj535:head 2026-02-11T23:31:17.635+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9f7e105e:::obj535:head v 107'535 2026-02-11T23:31:17.635+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.635+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.635+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9f7e105e:::obj535:head, version: 107'535, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f7e105e:::obj535:head@107'535, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.635+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9f41ce81:::obj424:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.635+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9f41ce81:::obj424:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 115/115 lb 4:9f41ce81:::obj424:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.635+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.635+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9f41ce81:::obj424:head) v3 2026-02-11T23:31:17.635+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9c9438f5:::obj513:head local-lis/les=112/113 n=364 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9f41ce81:::obj424:head) 2026-02-11T23:31:17.635+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.635+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9f41ce81:::obj424:head) v3 finish latency 0.000015 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.635+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.635+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.635+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3980 con 0x3e4a000 2026-02-11T23:31:17.635+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9f7e105e:::obj535:head)]) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.635+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.636+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 779 ==== MOSDPGPush(4.0 115/112 [PushOp(4:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3e4a000 2026-02-11T23:31:17.636+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.636+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.636+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 780 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9f7e105e:::obj535:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3e4a000 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.636+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9f7e105e:::obj535:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.636+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9f7e105e:::obj535:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9f7e105e:::obj535:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:9f8e934c:::obj456:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:9f8e934c:::obj456:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:9f8e934c:::obj456:head 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:9f8e934c:::obj456:head v 107'456 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.636+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:9f8e934c:::obj456:head, version: 107'456, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:9f8e934c:::obj456:head@107'456, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9f7e105e:::obj535:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9f7e105e:::obj535:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 115/115 lb 4:9f7e105e:::obj535:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9f7e105e:::obj535:head) v3 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f41ce81:::obj424:head local-lis/les=112/113 n=365 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9f7e105e:::obj535:head) 2026-02-11T23:31:17.636+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9f7e105e:::obj535:head) v3 finish latency 0.000037 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.636+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3b00 con 0x3e4a000 2026-02-11T23:31:17.636+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:9f8e934c:::obj456:head)]) -- 0x4c1fc80 con 0x3e4a000 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.636+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 781 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3e4a000 2026-02-11T23:31:17.636+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.636+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.636+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 782 ==== pg_backfill(progress 4.0 e 115/115 lb 4:9f8e934c:::obj456:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3e4a000 2026-02-11T23:31:17.636+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:9f8e934c:::obj456:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.636+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9f8e934c:::obj456:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9f8e934c:::obj456:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a0060720:::obj493:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a0060720:::obj493:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a0060720:::obj493:head 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a0060720:::obj493:head v 107'493 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.636+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.636+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a0060720:::obj493:head, version: 107'493, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0060720:::obj493:head@107'493, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:9f8e934c:::obj456:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9f8e934c:::obj456:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 115/115 lb 4:9f8e934c:::obj456:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:9f8e934c:::obj456:head) v3 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f7e105e:::obj535:head local-lis/les=112/113 n=366 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:9f8e934c:::obj456:head) 2026-02-11T23:31:17.636+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:9f8e934c:::obj456:head) v3 finish latency 0.000013 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.636+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.636+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.636+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3c80 con 0x3e4a000 2026-02-11T23:31:17.636+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a0060720:::obj493:head)]) -- 0x4c2de00 con 0x3e4a000 2026-02-11T23:31:17.636+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.637+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 783 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3e4a000 2026-02-11T23:31:17.637+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.637+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.637+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 784 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a0060720:::obj493:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5bb80 con 0x3e4a000 2026-02-11T23:31:17.637+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a0060720:::obj493:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.637+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a0060720:::obj493:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.637+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.637+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.637+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.637+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a0060720:::obj493:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.637+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.637+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.637+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a0217245:::obj31:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a0217245:::obj31:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.637+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a0217245:::obj31:head 2026-02-11T23:31:17.637+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a0217245:::obj31:head v 107'31 2026-02-11T23:31:17.637+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.637+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.637+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a0217245:::obj31:head, version: 107'31, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0217245:::obj31:head@107'31, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000067 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.637+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.637+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a0060720:::obj493:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a0060720:::obj493:head) v3 prio 15 cost 0 latency 0.000090 pg_backfill(progress 4.0 e 115/115 lb 4:a0060720:::obj493:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a0060720:::obj493:head) v3 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:9f8e934c:::obj456:head local-lis/les=112/113 n=367 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a0060720:::obj493:head) 2026-02-11T23:31:17.637+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a0060720:::obj493:head) v3 finish latency 0.000016 2026-02-11T23:31:17.637+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.637+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.637+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.637+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3800 con 0x3e4a000 2026-02-11T23:31:17.637+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a0217245:::obj31:head)]) -- 0x53b3e00 con 0x3e4a000 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.637+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 785 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ebba0 con 0x3e4a000 2026-02-11T23:31:17.637+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.637+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.637+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.637+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 786 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a0217245:::obj31:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3e4a000 2026-02-11T23:31:17.637+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a0217245:::obj31:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.637+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a0217245:::obj31:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.637+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.637+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.637+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.637+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a0217245:::obj31:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a0e9da24:::obj323:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.637+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a0e9da24:::obj323:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a0e9da24:::obj323:head 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a0e9da24:::obj323:head v 107'323 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.637+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.637+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a0e9da24:::obj323:head, version: 107'323, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a0e9da24:::obj323:head@107'323, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.638+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a0217245:::obj31:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a0217245:::obj31:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 115/115 lb 4:a0217245:::obj31:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a0217245:::obj31:head) v3 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0060720:::obj493:head local-lis/les=112/113 n=368 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a0217245:::obj31:head) 2026-02-11T23:31:17.638+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a0217245:::obj31:head) v3 finish latency 0.000014 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.638+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.638+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.638+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a0e9da24:::obj323:head)]) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.638+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 787 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.638+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.638+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.638+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 788 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a0e9da24:::obj323:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3e4a000 2026-02-11T23:31:17.638+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a0e9da24:::obj323:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.638+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a0e9da24:::obj323:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.638+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.638+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a0e9da24:::obj323:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a10cad2d:::obj340:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a10cad2d:::obj340:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a10cad2d:::obj340:head 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a10cad2d:::obj340:head v 107'340 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.638+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a10cad2d:::obj340:head, version: 107'340, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a10cad2d:::obj340:head@107'340, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.638+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.638+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a0e9da24:::obj323:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.638+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a0e9da24:::obj323:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:a0e9da24:::obj323:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.638+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.638+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a0e9da24:::obj323:head) v3 2026-02-11T23:31:17.638+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0217245:::obj31:head local-lis/les=112/113 n=369 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a0e9da24:::obj323:head) 2026-02-11T23:31:17.638+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.638+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a0e9da24:::obj323:head) v3 finish latency 0.000016 2026-02-11T23:31:17.638+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.638+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.638+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.638+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3500 con 0x3e4a000 2026-02-11T23:31:17.638+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a10cad2d:::obj340:head)]) -- 0x53b3680 con 0x3e4a000 2026-02-11T23:31:17.638+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.639+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 789 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.639+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.639+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.639+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 790 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a10cad2d:::obj340:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3e4a000 2026-02-11T23:31:17.639+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a10cad2d:::obj340:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.639+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a10cad2d:::obj340:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a10cad2d:::obj340:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a16a9a6a:::obj427:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a16a9a6a:::obj427:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a16a9a6a:::obj427:head 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a16a9a6a:::obj427:head v 107'427 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.639+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a16a9a6a:::obj427:head, version: 107'427, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16a9a6a:::obj427:head@107'427, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a10cad2d:::obj340:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a10cad2d:::obj340:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 115/115 lb 4:a10cad2d:::obj340:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a10cad2d:::obj340:head) v3 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a0e9da24:::obj323:head local-lis/les=112/113 n=370 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a10cad2d:::obj340:head) 2026-02-11T23:31:17.639+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a10cad2d:::obj340:head) v3 finish latency 0.000012 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.639+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4f50180 con 0x3e4a000 2026-02-11T23:31:17.639+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a16a9a6a:::obj427:head)]) -- 0x4c1f380 con 0x3e4a000 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.639+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 791 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.639+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.639+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.639+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 792 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a16a9a6a:::obj427:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3e4a000 2026-02-11T23:31:17.639+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a16a9a6a:::obj427:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.639+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a16a9a6a:::obj427:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a16a9a6a:::obj427:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a16dc95c:::obj269:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a16dc95c:::obj269:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a16dc95c:::obj269:head 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a16dc95c:::obj269:head v 107'269 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.639+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.639+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a16dc95c:::obj269:head, version: 107'269, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a16dc95c:::obj269:head@107'269, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.639+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a16a9a6a:::obj427:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a16a9a6a:::obj427:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:a16a9a6a:::obj427:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a16a9a6a:::obj427:head) v3 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a10cad2d:::obj340:head local-lis/les=112/113 n=371 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a16a9a6a:::obj427:head) 2026-02-11T23:31:17.639+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.639+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a16a9a6a:::obj427:head) v3 finish latency 0.000016 2026-02-11T23:31:17.639+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.640+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.640+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3200 con 0x3e4a000 2026-02-11T23:31:17.640+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a16dc95c:::obj269:head)]) -- 0x53b3380 con 0x3e4a000 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.640+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 793 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.640+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.640+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.640+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 794 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a16dc95c:::obj269:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3e4a000 2026-02-11T23:31:17.640+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a16dc95c:::obj269:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.640+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a16dc95c:::obj269:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.640+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a16dc95c:::obj269:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.640+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.640+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.640+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.640+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a22120c4:::obj223:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a22120c4:::obj223:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.640+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a22120c4:::obj223:head 2026-02-11T23:31:17.640+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a22120c4:::obj223:head v 107'223 2026-02-11T23:31:17.640+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.640+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.640+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a22120c4:::obj223:head, version: 107'223, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a22120c4:::obj223:head@107'223, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a16dc95c:::obj269:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a16dc95c:::obj269:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:a16dc95c:::obj269:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a16dc95c:::obj269:head) v3 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16a9a6a:::obj427:head local-lis/les=112/113 n=372 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a16dc95c:::obj269:head) 2026-02-11T23:31:17.640+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a16dc95c:::obj269:head) v3 finish latency 0.000015 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.640+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.640+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4780 con 0x3e4a000 2026-02-11T23:31:17.640+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a22120c4:::obj223:head)]) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.640+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 795 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43f2680 con 0x3e4a000 2026-02-11T23:31:17.640+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.640+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.640+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 796 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a22120c4:::obj223:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3e4a000 2026-02-11T23:31:17.640+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a22120c4:::obj223:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.640+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a22120c4:::obj223:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.640+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a22120c4:::obj223:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a25cd2bd:::obj42:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.640+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a25cd2bd:::obj42:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a25cd2bd:::obj42:head 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a25cd2bd:::obj42:head v 107'42 2026-02-11T23:31:17.640+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.641+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.641+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a25cd2bd:::obj42:head, version: 107'42, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a25cd2bd:::obj42:head@107'42, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.641+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a22120c4:::obj223:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a22120c4:::obj223:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:a22120c4:::obj223:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a22120c4:::obj223:head) v3 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a16dc95c:::obj269:head local-lis/les=112/113 n=373 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a22120c4:::obj223:head) 2026-02-11T23:31:17.641+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a22120c4:::obj223:head) v3 finish latency 0.000012 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.641+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.641+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2f00 con 0x3e4a000 2026-02-11T23:31:17.641+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a25cd2bd:::obj42:head)]) -- 0x511bb00 con 0x3e4a000 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.641+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 797 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3e4a000 2026-02-11T23:31:17.641+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.641+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.641+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 798 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a25cd2bd:::obj42:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3e4a000 2026-02-11T23:31:17.641+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a25cd2bd:::obj42:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.641+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a25cd2bd:::obj42:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.641+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.641+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a25cd2bd:::obj42:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 115/112 [PushOp(4:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a2bd80aa:::obj577:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a2bd80aa:::obj577:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a2bd80aa:::obj577:head 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a2bd80aa:::obj577:head v 107'577 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.641+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a2bd80aa:::obj577:head, version: 107'577, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a2bd80aa:::obj577:head@107'577, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.641+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.641+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a25cd2bd:::obj42:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.641+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a25cd2bd:::obj42:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 115/115 lb 4:a25cd2bd:::obj42:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.641+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.641+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a25cd2bd:::obj42:head) v3 2026-02-11T23:31:17.641+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a22120c4:::obj223:head local-lis/les=112/113 n=374 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a25cd2bd:::obj42:head) 2026-02-11T23:31:17.641+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.641+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a25cd2bd:::obj42:head) v3 finish latency 0.000015 2026-02-11T23:31:17.641+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.641+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.641+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.641+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4600 con 0x3e4a000 2026-02-11T23:31:17.641+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a2bd80aa:::obj577:head)]) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.641+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.642+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 799 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51824e0 con 0x3e4a000 2026-02-11T23:31:17.642+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.642+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.642+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 800 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a2bd80aa:::obj577:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3e4a000 2026-02-11T23:31:17.642+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a2bd80aa:::obj577:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.642+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a2bd80aa:::obj577:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a2bd80aa:::obj577:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a3278361:::obj425:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a3278361:::obj425:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a3278361:::obj425:head 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a3278361:::obj425:head v 107'425 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.642+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a3278361:::obj425:head, version: 107'425, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3278361:::obj425:head@107'425, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a2bd80aa:::obj577:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.642+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a2bd80aa:::obj577:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:a2bd80aa:::obj577:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.642+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.642+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a2bd80aa:::obj577:head) v3 2026-02-11T23:31:17.642+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a25cd2bd:::obj42:head local-lis/les=112/113 n=375 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a2bd80aa:::obj577:head) 2026-02-11T23:31:17.642+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.642+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a2bd80aa:::obj577:head) v3 finish latency 0.000012 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.642+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.642+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2c00 con 0x3e4a000 2026-02-11T23:31:17.642+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a3278361:::obj425:head)]) -- 0x4bf1200 con 0x3e4a000 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.642+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 801 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182000 con 0x3e4a000 2026-02-11T23:31:17.642+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.642+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.642+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 802 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a3278361:::obj425:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3e4a000 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.642+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a3278361:::obj425:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.642+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a3278361:::obj425:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.642+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a3278361:::obj425:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a3411a0c:::obj160:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.642+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a3411a0c:::obj160:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a3411a0c:::obj160:head 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a3411a0c:::obj160:head v 107'160 2026-02-11T23:31:17.642+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.642+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.643+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a3411a0c:::obj160:head, version: 107'160, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3411a0c:::obj160:head@107'160, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.643+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a3278361:::obj425:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a3278361:::obj425:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:a3278361:::obj425:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a3278361:::obj425:head) v3 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a2bd80aa:::obj577:head local-lis/les=112/113 n=376 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a3278361:::obj425:head) 2026-02-11T23:31:17.643+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a3278361:::obj425:head) v3 finish latency 0.000010 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.643+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.643+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.643+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a3411a0c:::obj160:head)]) -- 0x4bf1080 con 0x3e4a000 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.643+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 803 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516da00 con 0x3e4a000 2026-02-11T23:31:17.643+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.643+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.643+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 804 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a3411a0c:::obj160:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3e4a000 2026-02-11T23:31:17.643+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a3411a0c:::obj160:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.643+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a3411a0c:::obj160:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.643+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.643+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a3411a0c:::obj160:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a37ff878:::obj107:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a37ff878:::obj107:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a37ff878:::obj107:head 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a37ff878:::obj107:head v 107'107 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.643+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a37ff878:::obj107:head, version: 107'107, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a37ff878:::obj107:head@107'107, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.643+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.643+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a3411a0c:::obj160:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.643+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a3411a0c:::obj160:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:a3411a0c:::obj160:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.643+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.643+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a3411a0c:::obj160:head) v3 2026-02-11T23:31:17.643+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3278361:::obj425:head local-lis/les=112/113 n=377 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a3411a0c:::obj160:head) 2026-02-11T23:31:17.643+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.643+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a3411a0c:::obj160:head) v3 finish latency 0.000018 2026-02-11T23:31:17.643+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.643+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.643+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.643+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2900 con 0x3e4a000 2026-02-11T23:31:17.643+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a37ff878:::obj107:head)]) -- 0x53b2a80 con 0x3e4a000 2026-02-11T23:31:17.643+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.644+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 805 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4de4680 con 0x3e4a000 2026-02-11T23:31:17.644+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.644+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.644+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 806 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a37ff878:::obj107:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bc100 con 0x3e4a000 2026-02-11T23:31:17.644+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a37ff878:::obj107:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.644+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a37ff878:::obj107:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.644+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.644+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a37ff878:::obj107:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a383681b:::obj41:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a383681b:::obj41:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a383681b:::obj41:head 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a383681b:::obj41:head v 107'41 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.644+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a383681b:::obj41:head, version: 107'41, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a383681b:::obj41:head@107'41, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.644+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.644+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a37ff878:::obj107:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.644+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a37ff878:::obj107:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:a37ff878:::obj107:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.644+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.644+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a37ff878:::obj107:head) v3 2026-02-11T23:31:17.644+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3411a0c:::obj160:head local-lis/les=112/113 n=378 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a37ff878:::obj107:head) 2026-02-11T23:31:17.644+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.644+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a37ff878:::obj107:head) v3 finish latency 0.000016 2026-02-11T23:31:17.644+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.644+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.644+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2780 con 0x3e4a000 2026-02-11T23:31:17.644+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a383681b:::obj41:head)]) -- 0x4be9200 con 0x3e4a000 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.644+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 807 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183040 con 0x3e4a000 2026-02-11T23:31:17.644+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.644+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.644+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 808 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a383681b:::obj41:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3e4a000 2026-02-11T23:31:17.644+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a383681b:::obj41:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.644+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a383681b:::obj41:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.644+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.644+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a383681b:::obj41:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a39a40f0:::obj561:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a39a40f0:::obj561:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a39a40f0:::obj561:head 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a39a40f0:::obj561:head v 107'561 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.644+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.644+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a39a40f0:::obj561:head, version: 107'561, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a39a40f0:::obj561:head@107'561, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.644+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.644+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.644+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a383681b:::obj41:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.644+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a383681b:::obj41:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:a383681b:::obj41:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.644+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.644+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a383681b:::obj41:head) v3 2026-02-11T23:31:17.644+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a37ff878:::obj107:head local-lis/les=112/113 n=379 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a383681b:::obj41:head) 2026-02-11T23:31:17.644+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.644+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a383681b:::obj41:head) v3 finish latency 0.000015 2026-02-11T23:31:17.644+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.645+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4be9800 con 0x3e4a000 2026-02-11T23:31:17.645+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a39a40f0:::obj561:head)]) -- 0x4be9500 con 0x3e4a000 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.645+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 809 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.645+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.645+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.645+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 810 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a39a40f0:::obj561:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.645+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a39a40f0:::obj561:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.645+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a39a40f0:::obj561:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a39a40f0:::obj561:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a3cf8cd5:::obj363:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a3cf8cd5:::obj363:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a3cf8cd5:::obj363:head 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a3cf8cd5:::obj363:head v 107'363 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.645+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a3cf8cd5:::obj363:head, version: 107'363, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a3cf8cd5:::obj363:head@107'363, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a39a40f0:::obj561:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a39a40f0:::obj561:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:a39a40f0:::obj561:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a39a40f0:::obj561:head) v3 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a383681b:::obj41:head local-lis/les=112/113 n=380 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a39a40f0:::obj561:head) 2026-02-11T23:31:17.645+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a39a40f0:::obj561:head) v3 finish latency 0.000016 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.645+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2300 con 0x3e4a000 2026-02-11T23:31:17.645+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a3cf8cd5:::obj363:head)]) -- 0x53b2480 con 0x3e4a000 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.645+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 811 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.645+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.645+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.645+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 812 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a3cf8cd5:::obj363:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3e4a000 2026-02-11T23:31:17.645+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a3cf8cd5:::obj363:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.645+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a3cf8cd5:::obj363:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a3cf8cd5:::obj363:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a42b336e:::obj435:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a42b336e:::obj435:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a42b336e:::obj435:head 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a42b336e:::obj435:head v 107'435 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.645+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.645+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a42b336e:::obj435:head, version: 107'435, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a42b336e:::obj435:head@107'435, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.645+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a3cf8cd5:::obj363:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a3cf8cd5:::obj363:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 115/115 lb 4:a3cf8cd5:::obj363:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a3cf8cd5:::obj363:head) v3 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a39a40f0:::obj561:head local-lis/les=112/113 n=381 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a3cf8cd5:::obj363:head) 2026-02-11T23:31:17.645+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.645+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a3cf8cd5:::obj363:head) v3 finish latency 0.000013 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.645+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.646+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.646+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.646+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4be9e00 con 0x3e4a000 2026-02-11T23:31:17.646+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a42b336e:::obj435:head)]) -- 0x4bf0f00 con 0x3e4a000 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.646+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 813 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.646+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.646+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.646+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.646+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 814 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a42b336e:::obj435:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.646+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.646+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a42b336e:::obj435:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 115 2026-02-11T23:31:17.646+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a42b336e:::obj435:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.646+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.646+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.646+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.646+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a42b336e:::obj435:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.646+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.646+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.646+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a4383432:::obj559:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.646+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a4383432:::obj559:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.646+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a4383432:::obj559:head 2026-02-11T23:31:17.646+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a4383432:::obj559:head v 107'559 2026-02-11T23:31:17.646+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.646+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.646+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a4383432:::obj559:head, version: 107'559, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4383432:::obj559:head@107'559, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.646+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a42b336e:::obj435:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.646+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a42b336e:::obj435:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 115/115 lb 4:a42b336e:::obj435:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.646+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.646+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a42b336e:::obj435:head) v3 2026-02-11T23:31:17.646+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a3cf8cd5:::obj363:head local-lis/les=112/113 n=382 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a42b336e:::obj435:head) 2026-02-11T23:31:17.646+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.646+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a42b336e:::obj435:head) v3 finish latency 0.000010 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.646+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.646+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.646+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2180 con 0x3e4a000 2026-02-11T23:31:17.646+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a4383432:::obj559:head)]) -- 0x4bf0d80 con 0x3e4a000 2026-02-11T23:31:17.646+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.647+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 815 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.647+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.647+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.647+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 816 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a4383432:::obj559:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.647+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a4383432:::obj559:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.647+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a4383432:::obj559:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a4383432:::obj559:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a4518626:::obj581:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a4518626:::obj581:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a4518626:::obj581:head 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a4518626:::obj581:head v 107'581 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.647+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a4518626:::obj581:head, version: 107'581, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4518626:::obj581:head@107'581, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a4383432:::obj559:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a4383432:::obj559:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:a4383432:::obj559:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a4383432:::obj559:head) v3 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a42b336e:::obj435:head local-lis/les=112/113 n=383 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a4383432:::obj559:head) 2026-02-11T23:31:17.647+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a4383432:::obj559:head) v3 finish latency 0.000016 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.647+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4be9200 con 0x3e4a000 2026-02-11T23:31:17.647+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a4518626:::obj581:head)]) -- 0x4ba2d80 con 0x3e4a000 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.647+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 817 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3e4a000 2026-02-11T23:31:17.647+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.647+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.647+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 818 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a4518626:::obj581:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.647+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a4518626:::obj581:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.647+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a4518626:::obj581:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a4518626:::obj581:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a4664b16:::obj367:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a4664b16:::obj367:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a4664b16:::obj367:head 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a4664b16:::obj367:head v 107'367 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.647+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.647+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a4664b16:::obj367:head, version: 107'367, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a4664b16:::obj367:head@107'367, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a4518626:::obj581:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a4518626:::obj581:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 115/115 lb 4:a4518626:::obj581:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a4518626:::obj581:head) v3 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4383432:::obj559:head local-lis/les=112/113 n=384 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a4518626:::obj581:head) 2026-02-11T23:31:17.647+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a4518626:::obj581:head) v3 finish latency 0.000013 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.647+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.647+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.647+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2780 con 0x3e4a000 2026-02-11T23:31:17.647+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a4664b16:::obj367:head)]) -- 0x4bf0c00 con 0x3e4a000 2026-02-11T23:31:17.647+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.648+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 819 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3e4a000 2026-02-11T23:31:17.648+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.648+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.648+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.648+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 820 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a4664b16:::obj367:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.648+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a4664b16:::obj367:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.648+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.648+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a4664b16:::obj367:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.648+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.648+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.648+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.648+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.648+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.648+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a4664b16:::obj367:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.648+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.648+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.648+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a52fdb40:::obj548:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.648+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a52fdb40:::obj548:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.648+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a52fdb40:::obj548:head 2026-02-11T23:31:17.648+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a52fdb40:::obj548:head v 107'548 2026-02-11T23:31:17.648+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.648+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.648+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a52fdb40:::obj548:head, version: 107'548, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a52fdb40:::obj548:head@107'548, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.648+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.648+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.648+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.648+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a4664b16:::obj367:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.648+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a4664b16:::obj367:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:a4664b16:::obj367:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.648+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.648+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a4664b16:::obj367:head) v3 2026-02-11T23:31:17.648+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4518626:::obj581:head local-lis/les=112/113 n=385 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a4664b16:::obj367:head) 2026-02-11T23:31:17.648+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.648+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a4664b16:::obj367:head) v3 finish latency 0.000016 2026-02-11T23:31:17.648+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.649+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.649+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.649+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1080 con 0x3e4a000 2026-02-11T23:31:17.649+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a52fdb40:::obj548:head)]) -- 0x53b2a80 con 0x3e4a000 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.649+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 821 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.649+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.649+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.649+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.649+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 822 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a52fdb40:::obj548:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.649+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a52fdb40:::obj548:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.649+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.649+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a52fdb40:::obj548:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.649+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.649+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.649+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a52fdb40:::obj548:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.649+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.649+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.649+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.649+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a53b2263:::obj137:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.649+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a53b2263:::obj137:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.649+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a53b2263:::obj137:head 2026-02-11T23:31:17.649+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a53b2263:::obj137:head v 107'137 2026-02-11T23:31:17.649+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.649+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.649+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a53b2263:::obj137:head, version: 107'137, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a53b2263:::obj137:head@107'137, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.649+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a52fdb40:::obj548:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.649+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a52fdb40:::obj548:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:a52fdb40:::obj548:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.649+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.649+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a52fdb40:::obj548:head) v3 2026-02-11T23:31:17.649+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a4664b16:::obj367:head local-lis/les=112/113 n=386 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a52fdb40:::obj548:head) 2026-02-11T23:31:17.649+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.649+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a52fdb40:::obj548:head) v3 finish latency 0.000012 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.649+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.649+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.649+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.649+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a53b2263:::obj137:head)]) -- 0x4bf0a80 con 0x3e4a000 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.649+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 823 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3e4a000 2026-02-11T23:31:17.649+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.649+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.649+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 824 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a53b2263:::obj137:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.649+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a53b2263:::obj137:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.649+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a53b2263:::obj137:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.649+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a53b2263:::obj137:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a555be2b:::obj75:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a555be2b:::obj75:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a555be2b:::obj75:head 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a555be2b:::obj75:head v 107'75 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.650+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a555be2b:::obj75:head, version: 107'75, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a555be2b:::obj75:head@107'75, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a53b2263:::obj137:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a53b2263:::obj137:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 115/115 lb 4:a53b2263:::obj137:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a53b2263:::obj137:head) v3 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a52fdb40:::obj548:head local-lis/les=112/113 n=387 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a53b2263:::obj137:head) 2026-02-11T23:31:17.650+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a53b2263:::obj137:head) v3 finish latency 0.000015 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.650+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2c00 con 0x3e4a000 2026-02-11T23:31:17.650+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a555be2b:::obj75:head)]) -- 0x4bf1200 con 0x3e4a000 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.650+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 825 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.650+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.650+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.650+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 826 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a555be2b:::obj75:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.650+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a555be2b:::obj75:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.650+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a555be2b:::obj75:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a555be2b:::obj75:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a557efb1:::obj8:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a557efb1:::obj8:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a557efb1:::obj8:head 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a557efb1:::obj8:head v 107'8 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.650+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.650+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a557efb1:::obj8:head, version: 107'8, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a557efb1:::obj8:head@107'8, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a555be2b:::obj75:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a555be2b:::obj75:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:a555be2b:::obj75:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a555be2b:::obj75:head) v3 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a53b2263:::obj137:head local-lis/les=112/113 n=388 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a555be2b:::obj75:head) 2026-02-11T23:31:17.650+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a555be2b:::obj75:head) v3 finish latency 0.000012 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.650+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.650+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.650+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.650+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a557efb1:::obj8:head)]) -- 0x4bf0900 con 0x3e4a000 2026-02-11T23:31:17.650+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.651+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 827 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3e4a000 2026-02-11T23:31:17.651+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.651+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.651+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 828 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a557efb1:::obj8:head) ==== 1060+0+0 (unknown 0 0 0) 0x5165700 con 0x3e4a000 2026-02-11T23:31:17.651+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a557efb1:::obj8:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.651+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a557efb1:::obj8:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a557efb1:::obj8:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a5c2f6e2:::obj111:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a5c2f6e2:::obj111:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a5c2f6e2:::obj111:head 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a5c2f6e2:::obj111:head v 107'111 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.651+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a5c2f6e2:::obj111:head, version: 107'111, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5c2f6e2:::obj111:head@107'111, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a557efb1:::obj8:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a557efb1:::obj8:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:a557efb1:::obj8:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a557efb1:::obj8:head) v3 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a555be2b:::obj75:head local-lis/les=112/113 n=389 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a557efb1:::obj8:head) 2026-02-11T23:31:17.651+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a557efb1:::obj8:head) v3 finish latency 0.000015 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.651+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2f00 con 0x3e4a000 2026-02-11T23:31:17.651+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a5c2f6e2:::obj111:head)]) -- 0x4ae4600 con 0x3e4a000 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.651+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 829 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3e4a000 2026-02-11T23:31:17.651+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.651+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.651+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 830 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a5c2f6e2:::obj111:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165180 con 0x3e4a000 2026-02-11T23:31:17.651+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a5c2f6e2:::obj111:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.651+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a5c2f6e2:::obj111:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a5c2f6e2:::obj111:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a5ea1435:::obj539:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a5ea1435:::obj539:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a5ea1435:::obj539:head 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a5ea1435:::obj539:head v 107'539 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.651+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.651+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a5ea1435:::obj539:head, version: 107'539, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a5ea1435:::obj539:head@107'539, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.651+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a5c2f6e2:::obj111:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a5c2f6e2:::obj111:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 115/115 lb 4:a5c2f6e2:::obj111:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a5c2f6e2:::obj111:head) v3 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a557efb1:::obj8:head local-lis/les=112/113 n=390 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a5c2f6e2:::obj111:head) 2026-02-11T23:31:17.651+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.651+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a5c2f6e2:::obj111:head) v3 finish latency 0.000019 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.651+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.652+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4e5c480 con 0x3e4a000 2026-02-11T23:31:17.652+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a5ea1435:::obj539:head)]) -- 0x4ae4180 con 0x3e4a000 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.652+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 831 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.652+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.652+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.652+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 832 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a5ea1435:::obj539:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164c00 con 0x3e4a000 2026-02-11T23:31:17.652+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a5ea1435:::obj539:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.652+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a5ea1435:::obj539:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a5ea1435:::obj539:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a601bbe9:::obj562:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a601bbe9:::obj562:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a601bbe9:::obj562:head 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a601bbe9:::obj562:head v 107'562 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.652+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a601bbe9:::obj562:head, version: 107'562, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a601bbe9:::obj562:head@107'562, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a5ea1435:::obj539:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.652+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a5ea1435:::obj539:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:a5ea1435:::obj539:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.652+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.652+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a5ea1435:::obj539:head) v3 2026-02-11T23:31:17.652+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5c2f6e2:::obj111:head local-lis/les=112/113 n=391 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a5ea1435:::obj539:head) 2026-02-11T23:31:17.652+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.652+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a5ea1435:::obj539:head) v3 finish latency 0.000016 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.652+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3200 con 0x3e4a000 2026-02-11T23:31:17.652+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a601bbe9:::obj562:head)]) -- 0x4ae4780 con 0x3e4a000 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.652+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 833 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.652+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.652+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.652+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 834 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a601bbe9:::obj562:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3e4a000 2026-02-11T23:31:17.652+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a601bbe9:::obj562:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.652+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a601bbe9:::obj562:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a601bbe9:::obj562:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a61ad63e:::obj13:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a61ad63e:::obj13:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a61ad63e:::obj13:head 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a61ad63e:::obj13:head v 107'13 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.652+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.652+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a61ad63e:::obj13:head, version: 107'13, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a61ad63e:::obj13:head@107'13, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.652+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a601bbe9:::obj562:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.652+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a601bbe9:::obj562:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:a601bbe9:::obj562:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.652+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.652+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a601bbe9:::obj562:head) v3 2026-02-11T23:31:17.652+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a5ea1435:::obj539:head local-lis/les=112/113 n=392 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a601bbe9:::obj562:head) 2026-02-11T23:31:17.652+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.652+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a601bbe9:::obj562:head) v3 finish latency 0.000015 2026-02-11T23:31:17.652+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.653+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.653+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.653+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4f50180 con 0x3e4a000 2026-02-11T23:31:17.653+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a61ad63e:::obj13:head)]) -- 0x4c1f380 con 0x3e4a000 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.653+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 835 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3a00 con 0x3e4a000 2026-02-11T23:31:17.653+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.653+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.653+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.653+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 836 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a61ad63e:::obj13:head) ==== 1061+0+0 (unknown 0 0 0) 0x4def180 con 0x3e4a000 2026-02-11T23:31:17.653+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a61ad63e:::obj13:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.653+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.653+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a61ad63e:::obj13:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.653+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.653+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.653+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.653+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a61ad63e:::obj13:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.653+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.653+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.653+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a630f391:::obj375:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.653+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a630f391:::obj375:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.653+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a630f391:::obj375:head 2026-02-11T23:31:17.653+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a630f391:::obj375:head v 107'375 2026-02-11T23:31:17.653+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.653+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.653+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a630f391:::obj375:head, version: 107'375, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a630f391:::obj375:head@107'375, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.653+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a61ad63e:::obj13:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.653+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a61ad63e:::obj13:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:a61ad63e:::obj13:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.653+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.653+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a61ad63e:::obj13:head) v3 2026-02-11T23:31:17.653+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a601bbe9:::obj562:head local-lis/les=112/113 n=393 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a61ad63e:::obj13:head) 2026-02-11T23:31:17.653+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.653+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a61ad63e:::obj13:head) v3 finish latency 0.000012 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.653+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.653+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.653+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3680 con 0x3e4a000 2026-02-11T23:31:17.653+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a630f391:::obj375:head)]) -- 0x57de180 con 0x3e4a000 2026-02-11T23:31:17.653+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.654+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 837 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.654+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.654+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.654+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 838 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a630f391:::obj375:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd180 con 0x3e4a000 2026-02-11T23:31:17.654+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a630f391:::obj375:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.654+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a630f391:::obj375:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a630f391:::obj375:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a6b27c5f:::obj420:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a6b27c5f:::obj420:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a6b27c5f:::obj420:head 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a6b27c5f:::obj420:head v 107'420 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.654+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a6b27c5f:::obj420:head, version: 107'420, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6b27c5f:::obj420:head@107'420, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a630f391:::obj375:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a630f391:::obj375:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:a630f391:::obj375:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a630f391:::obj375:head) v3 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a61ad63e:::obj13:head local-lis/les=112/113 n=394 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a630f391:::obj375:head) 2026-02-11T23:31:17.654+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a630f391:::obj375:head) v3 finish latency 0.000010 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.654+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3500 con 0x3e4a000 2026-02-11T23:31:17.654+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a6b27c5f:::obj420:head)]) -- 0x57de480 con 0x3e4a000 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.654+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 839 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3e4a000 2026-02-11T23:31:17.654+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.654+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.654+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 840 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a6b27c5f:::obj420:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3e4a000 2026-02-11T23:31:17.654+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a6b27c5f:::obj420:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.654+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a6b27c5f:::obj420:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a6b27c5f:::obj420:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a6f174aa:::obj203:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a6f174aa:::obj203:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a6f174aa:::obj203:head 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a6f174aa:::obj203:head v 107'203 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.654+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.654+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a6f174aa:::obj203:head, version: 107'203, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a6f174aa:::obj203:head@107'203, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a6b27c5f:::obj420:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a6b27c5f:::obj420:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:a6b27c5f:::obj420:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a6b27c5f:::obj420:head) v3 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a630f391:::obj375:head local-lis/les=112/113 n=395 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a6b27c5f:::obj420:head) 2026-02-11T23:31:17.654+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a6b27c5f:::obj420:head) v3 finish latency 0.000009 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.654+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.654+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.654+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.654+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a6f174aa:::obj203:head)]) -- 0x57de780 con 0x3e4a000 2026-02-11T23:31:17.654+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.655+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 841 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51831e0 con 0x3e4a000 2026-02-11T23:31:17.655+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.655+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.655+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 842 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a6f174aa:::obj203:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3e4a000 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.655+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a6f174aa:::obj203:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.655+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a6f174aa:::obj203:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.655+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.655+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a6f174aa:::obj203:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.655+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.655+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.655+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a8709cbe:::obj458:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a8709cbe:::obj458:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.655+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a8709cbe:::obj458:head 2026-02-11T23:31:17.655+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a8709cbe:::obj458:head v 107'458 2026-02-11T23:31:17.655+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.655+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.655+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a8709cbe:::obj458:head, version: 107'458, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8709cbe:::obj458:head@107'458, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a6f174aa:::obj203:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a6f174aa:::obj203:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:a6f174aa:::obj203:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a6f174aa:::obj203:head) v3 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6b27c5f:::obj420:head local-lis/les=112/113 n=396 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a6f174aa:::obj203:head) 2026-02-11T23:31:17.655+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a6f174aa:::obj203:head) v3 finish latency 0.000018 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.655+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.655+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3800 con 0x3e4a000 2026-02-11T23:31:17.655+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a8709cbe:::obj458:head)]) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.655+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 843 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3e4a000 2026-02-11T23:31:17.655+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.655+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.655+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 844 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a8709cbe:::obj458:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3e4a000 2026-02-11T23:31:17.655+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a8709cbe:::obj458:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.655+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a8709cbe:::obj458:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a8709cbe:::obj458:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a87532ae:::obj194:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a87532ae:::obj194:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a87532ae:::obj194:head 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a87532ae:::obj194:head v 107'194 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.655+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.655+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a87532ae:::obj194:head, version: 107'194, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a87532ae:::obj194:head@107'194, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.655+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.655+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a8709cbe:::obj458:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a8709cbe:::obj458:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 115/115 lb 4:a8709cbe:::obj458:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a8709cbe:::obj458:head) v3 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a6f174aa:::obj203:head local-lis/les=112/113 n=397 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a8709cbe:::obj458:head) 2026-02-11T23:31:17.656+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a8709cbe:::obj458:head) v3 finish latency 0.000020 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.656+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.656+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c2de00 con 0x3e4a000 2026-02-11T23:31:17.656+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a87532ae:::obj194:head)]) -- 0x57dea80 con 0x3e4a000 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.656+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 845 ==== MOSDPGPush(4.0 115/112 [PushOp(4:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51821a0 con 0x3e4a000 2026-02-11T23:31:17.656+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.656+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.656+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 846 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a87532ae:::obj194:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3e4a000 2026-02-11T23:31:17.656+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a87532ae:::obj194:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.656+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a87532ae:::obj194:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.656+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.656+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a87532ae:::obj194:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:a8d6b24d:::obj229:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:a8d6b24d:::obj229:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:a8d6b24d:::obj229:head 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:a8d6b24d:::obj229:head v 107'229 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.656+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:a8d6b24d:::obj229:head, version: 107'229, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:a8d6b24d:::obj229:head@107'229, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.656+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.656+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a87532ae:::obj194:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.656+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a87532ae:::obj194:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:a87532ae:::obj194:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.656+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.656+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a87532ae:::obj194:head) v3 2026-02-11T23:31:17.656+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8709cbe:::obj458:head local-lis/les=112/113 n=398 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a87532ae:::obj194:head) 2026-02-11T23:31:17.656+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.656+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a87532ae:::obj194:head) v3 finish latency 0.000015 2026-02-11T23:31:17.656+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.656+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.656+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.656+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3b00 con 0x3e4a000 2026-02-11T23:31:17.656+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:a8d6b24d:::obj229:head)]) -- 0x53b3c80 con 0x3e4a000 2026-02-11T23:31:17.656+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.657+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 847 ==== MOSDPGPush(4.0 115/112 [PushOp(4:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3e4a000 2026-02-11T23:31:17.657+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.657+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.657+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 848 ==== pg_backfill(progress 4.0 e 115/115 lb 4:a8d6b24d:::obj229:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded080 con 0x3e4a000 2026-02-11T23:31:17.657+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:a8d6b24d:::obj229:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.657+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a8d6b24d:::obj229:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a8d6b24d:::obj229:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:aa1e2680:::obj587:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:aa1e2680:::obj587:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:aa1e2680:::obj587:head 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:aa1e2680:::obj587:head v 107'587 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.657+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:aa1e2680:::obj587:head, version: 107'587, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aa1e2680:::obj587:head@107'587, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:a8d6b24d:::obj229:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a8d6b24d:::obj229:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:a8d6b24d:::obj229:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:a8d6b24d:::obj229:head) v3 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a87532ae:::obj194:head local-lis/les=112/113 n=399 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:a8d6b24d:::obj229:head) 2026-02-11T23:31:17.657+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:a8d6b24d:::obj229:head) v3 finish latency 0.000012 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.657+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.657+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:aa1e2680:::obj587:head)]) -- 0x57ded80 con 0x3e4a000 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.657+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 849 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3e4a000 2026-02-11T23:31:17.657+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.657+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.657+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 850 ==== pg_backfill(progress 4.0 e 115/115 lb 4:aa1e2680:::obj587:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3e4a000 2026-02-11T23:31:17.657+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:aa1e2680:::obj587:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.657+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:aa1e2680:::obj587:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:aa1e2680:::obj587:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ab0f5189:::obj85:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ab0f5189:::obj85:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ab0f5189:::obj85:head 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ab0f5189:::obj85:head v 107'85 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.657+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ab0f5189:::obj85:head, version: 107'85, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab0f5189:::obj85:head@107'85, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:aa1e2680:::obj587:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:aa1e2680:::obj587:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:aa1e2680:::obj587:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:aa1e2680:::obj587:head) v3 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:a8d6b24d:::obj229:head local-lis/les=112/113 n=400 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:aa1e2680:::obj587:head) 2026-02-11T23:31:17.657+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.657+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:aa1e2680:::obj587:head) v3 finish latency 0.000015 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.657+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.657+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.657+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.657+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ab0f5189:::obj85:head)]) -- 0x53b3980 con 0x3e4a000 2026-02-11T23:31:17.657+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.658+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 851 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.658+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.658+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.658+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 852 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ab0f5189:::obj85:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3e4a000 2026-02-11T23:31:17.658+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ab0f5189:::obj85:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.658+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ab0f5189:::obj85:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ab0f5189:::obj85:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ab946124:::obj15:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ab946124:::obj15:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ab946124:::obj15:head 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ab946124:::obj15:head v 107'15 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.658+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ab946124:::obj15:head, version: 107'15, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ab946124:::obj15:head@107'15, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ab0f5189:::obj85:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ab0f5189:::obj85:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:ab0f5189:::obj85:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ab0f5189:::obj85:head) v3 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aa1e2680:::obj587:head local-lis/les=112/113 n=401 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ab0f5189:::obj85:head) 2026-02-11T23:31:17.658+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ab0f5189:::obj85:head) v3 finish latency 0.000012 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.658+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.658+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ab946124:::obj15:head)]) -- 0x57df080 con 0x3e4a000 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.658+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 853 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3a00 con 0x3e4a000 2026-02-11T23:31:17.658+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.658+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.658+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 854 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ab946124:::obj15:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3e4a000 2026-02-11T23:31:17.658+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ab946124:::obj15:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.658+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ab946124:::obj15:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ab946124:::obj15:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ac7304c6:::obj483:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ac7304c6:::obj483:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ac7304c6:::obj483:head 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ac7304c6:::obj483:head v 107'483 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.658+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.658+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ac7304c6:::obj483:head, version: 107'483, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ac7304c6:::obj483:head@107'483, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.658+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ab946124:::obj15:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ab946124:::obj15:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:ab946124:::obj15:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ab946124:::obj15:head) v3 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab0f5189:::obj85:head local-lis/les=112/113 n=402 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ab946124:::obj15:head) 2026-02-11T23:31:17.658+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.658+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ab946124:::obj15:head) v3 finish latency 0.000015 2026-02-11T23:31:17.658+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.659+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.659+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57de180 con 0x3e4a000 2026-02-11T23:31:17.659+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ac7304c6:::obj483:head)]) -- 0x57de480 con 0x3e4a000 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.659+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 855 ==== MOSDPGPush(4.0 115/112 [PushOp(4:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.659+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.659+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.659+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 856 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ac7304c6:::obj483:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3e4a000 2026-02-11T23:31:17.659+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ac7304c6:::obj483:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.659+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ac7304c6:::obj483:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 115/112 [PushOp(4:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.659+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.659+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ac7304c6:::obj483:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:acb8a831:::obj238:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:acb8a831:::obj238:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:acb8a831:::obj238:head 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:acb8a831:::obj238:head v 107'238 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.659+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:acb8a831:::obj238:head, version: 107'238, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acb8a831:::obj238:head@107'238, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.659+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.659+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ac7304c6:::obj483:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.659+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ac7304c6:::obj483:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:ac7304c6:::obj483:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.659+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.659+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ac7304c6:::obj483:head) v3 2026-02-11T23:31:17.659+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ab946124:::obj15:head local-lis/les=112/113 n=403 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ac7304c6:::obj483:head) 2026-02-11T23:31:17.659+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.659+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ac7304c6:::obj483:head) v3 finish latency 0.000015 2026-02-11T23:31:17.659+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.659+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.659+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4f50180 con 0x3e4a000 2026-02-11T23:31:17.659+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:acb8a831:::obj238:head)]) -- 0x53b3680 con 0x3e4a000 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.659+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 857 ==== MOSDPGPush(4.0 115/112 [PushOp(4:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.659+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.659+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.659+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 858 ==== pg_backfill(progress 4.0 e 115/115 lb 4:acb8a831:::obj238:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3e4a000 2026-02-11T23:31:17.659+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:acb8a831:::obj238:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.659+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:acb8a831:::obj238:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.659+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.659+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:acb8a831:::obj238:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.659+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:acfc8dd9:::obj74:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.659+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:acfc8dd9:::obj74:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:acfc8dd9:::obj74:head 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:acfc8dd9:::obj74:head v 107'74 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.660+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:acfc8dd9:::obj74:head, version: 107'74, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:acfc8dd9:::obj74:head@107'74, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000065 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.660+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.660+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:acb8a831:::obj238:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.660+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:acb8a831:::obj238:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 115/115 lb 4:acb8a831:::obj238:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.660+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.660+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:acb8a831:::obj238:head) v3 2026-02-11T23:31:17.660+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ac7304c6:::obj483:head local-lis/les=112/113 n=404 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:acb8a831:::obj238:head) 2026-02-11T23:31:17.660+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.660+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:acb8a831:::obj238:head) v3 finish latency 0.000015 2026-02-11T23:31:17.660+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.660+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.660+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3200 con 0x3e4a000 2026-02-11T23:31:17.660+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:acfc8dd9:::obj74:head)]) -- 0x4ae4780 con 0x3e4a000 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.660+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 859 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3e4a000 2026-02-11T23:31:17.660+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.660+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.660+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 860 ==== pg_backfill(progress 4.0 e 115/115 lb 4:acfc8dd9:::obj74:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3e4a000 2026-02-11T23:31:17.660+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:acfc8dd9:::obj74:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.660+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:acfc8dd9:::obj74:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.660+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.660+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:acfc8dd9:::obj74:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ad830abd:::obj496:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ad830abd:::obj496:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ad830abd:::obj496:head 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ad830abd:::obj496:head v 107'496 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.660+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ad830abd:::obj496:head, version: 107'496, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ad830abd:::obj496:head@107'496, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.660+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.660+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:acfc8dd9:::obj74:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.660+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:acfc8dd9:::obj74:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:acfc8dd9:::obj74:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.660+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.660+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:acfc8dd9:::obj74:head) v3 2026-02-11T23:31:17.660+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acb8a831:::obj238:head local-lis/les=112/113 n=405 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:acfc8dd9:::obj74:head) 2026-02-11T23:31:17.660+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.660+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:acfc8dd9:::obj74:head) v3 finish latency 0.000015 2026-02-11T23:31:17.660+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.660+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.660+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.660+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4600 con 0x3e4a000 2026-02-11T23:31:17.660+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ad830abd:::obj496:head)]) -- 0x4ae4180 con 0x3e4a000 2026-02-11T23:31:17.660+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.661+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 861 ==== MOSDPGPush(4.0 115/112 [PushOp(4:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3e4a000 2026-02-11T23:31:17.661+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.661+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.661+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 862 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ad830abd:::obj496:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3e4a000 2026-02-11T23:31:17.661+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ad830abd:::obj496:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.661+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ad830abd:::obj496:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ad830abd:::obj496:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:adc1b1a4:::obj497:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:adc1b1a4:::obj497:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:adc1b1a4:::obj497:head 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:adc1b1a4:::obj497:head v 107'497 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.661+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:adc1b1a4:::obj497:head, version: 107'497, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:adc1b1a4:::obj497:head@107'497, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ad830abd:::obj496:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ad830abd:::obj496:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:ad830abd:::obj496:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ad830abd:::obj496:head) v3 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:acfc8dd9:::obj74:head local-lis/les=112/113 n=406 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ad830abd:::obj496:head) 2026-02-11T23:31:17.661+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ad830abd:::obj496:head) v3 finish latency 0.000012 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.661+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2f00 con 0x3e4a000 2026-02-11T23:31:17.661+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:adc1b1a4:::obj497:head)]) -- 0x57df200 con 0x3e4a000 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.661+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 863 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.661+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.661+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.661+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 864 ==== pg_backfill(progress 4.0 e 115/115 lb 4:adc1b1a4:::obj497:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3e4a000 2026-02-11T23:31:17.661+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:adc1b1a4:::obj497:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.661+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:adc1b1a4:::obj497:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:adc1b1a4:::obj497:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ae2ef588:::obj234:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ae2ef588:::obj234:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ae2ef588:::obj234:head 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ae2ef588:::obj234:head v 107'234 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.661+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.661+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ae2ef588:::obj234:head, version: 107'234, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ae2ef588:::obj234:head@107'234, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:adc1b1a4:::obj497:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:adc1b1a4:::obj497:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:adc1b1a4:::obj497:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:adc1b1a4:::obj497:head) v3 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ad830abd:::obj496:head local-lis/les=112/113 n=407 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:adc1b1a4:::obj497:head) 2026-02-11T23:31:17.661+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:adc1b1a4:::obj497:head) v3 finish latency 0.000011 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.661+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.661+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.661+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.662+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0900 con 0x3e4a000 2026-02-11T23:31:17.662+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ae2ef588:::obj234:head)]) -- 0x57df500 con 0x3e4a000 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.662+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 865 ==== MOSDPGPush(4.0 115/112 [PushOp(4:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3e4a000 2026-02-11T23:31:17.662+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.662+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.662+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 866 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ae2ef588:::obj234:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3e4a000 2026-02-11T23:31:17.662+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ae2ef588:::obj234:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.662+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ae2ef588:::obj234:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ae2ef588:::obj234:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:aec2b241:::obj347:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:aec2b241:::obj347:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:aec2b241:::obj347:head 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:aec2b241:::obj347:head v 107'347 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.662+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:aec2b241:::obj347:head, version: 107'347, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aec2b241:::obj347:head@107'347, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ae2ef588:::obj234:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ae2ef588:::obj234:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:ae2ef588:::obj234:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ae2ef588:::obj234:head) v3 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:adc1b1a4:::obj497:head local-lis/les=112/113 n=408 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ae2ef588:::obj234:head) 2026-02-11T23:31:17.662+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ae2ef588:::obj234:head) v3 finish latency 0.000010 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.662+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.662+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:aec2b241:::obj347:head)]) -- 0x57df800 con 0x3e4a000 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.662+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 867 ==== MOSDPGPush(4.0 115/112 [PushOp(4:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.662+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.662+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.662+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 868 ==== pg_backfill(progress 4.0 e 115/115 lb 4:aec2b241:::obj347:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3e4a000 2026-02-11T23:31:17.662+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:aec2b241:::obj347:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.662+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:aec2b241:::obj347:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:aec2b241:::obj347:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:aecaa835:::obj157:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:aecaa835:::obj157:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:aecaa835:::obj157:head 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:aecaa835:::obj157:head v 107'157 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.662+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.662+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:aecaa835:::obj157:head, version: 107'157, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:aecaa835:::obj157:head@107'157, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.662+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:aec2b241:::obj347:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:aec2b241:::obj347:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 115/115 lb 4:aec2b241:::obj347:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:aec2b241:::obj347:head) v3 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ae2ef588:::obj234:head local-lis/les=112/113 n=409 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:aec2b241:::obj347:head) 2026-02-11T23:31:17.662+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.662+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:aec2b241:::obj347:head) v3 finish latency 0.000013 2026-02-11T23:31:17.662+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.663+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.663+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.663+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0a80 con 0x3e4a000 2026-02-11T23:31:17.663+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:aecaa835:::obj157:head)]) -- 0x4bf1200 con 0x3e4a000 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.663+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 869 ==== MOSDPGPush(4.0 115/112 [PushOp(4:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3e4a000 2026-02-11T23:31:17.663+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.663+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.663+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.663+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 870 ==== pg_backfill(progress 4.0 e 115/115 lb 4:aecaa835:::obj157:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5080 con 0x3e4a000 2026-02-11T23:31:17.663+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:aecaa835:::obj157:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.663+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:aecaa835:::obj157:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.663+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.663+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.663+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.663+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.663+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:aecaa835:::obj157:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.663+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.663+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.663+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:af2b834d:::obj153:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.663+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:af2b834d:::obj153:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.663+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:af2b834d:::obj153:head 2026-02-11T23:31:17.663+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:af2b834d:::obj153:head v 107'153 2026-02-11T23:31:17.663+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.663+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.663+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af2b834d:::obj153:head, version: 107'153, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2b834d:::obj153:head@107'153, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.663+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:aecaa835:::obj157:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.663+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:aecaa835:::obj157:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 115/115 lb 4:aecaa835:::obj157:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.663+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.663+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:aecaa835:::obj157:head) v3 2026-02-11T23:31:17.663+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aec2b241:::obj347:head local-lis/les=112/113 n=410 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:aecaa835:::obj157:head) 2026-02-11T23:31:17.663+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.663+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:aecaa835:::obj157:head) v3 finish latency 0.000012 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.663+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.663+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.663+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.663+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:af2b834d:::obj153:head)]) -- 0x57dfb00 con 0x3e4a000 2026-02-11T23:31:17.663+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.664+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 871 ==== MOSDPGPush(4.0 115/112 [PushOp(4:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3e4a000 2026-02-11T23:31:17.664+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.664+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.664+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 872 ==== pg_backfill(progress 4.0 e 115/115 lb 4:af2b834d:::obj153:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3e4a000 2026-02-11T23:31:17.664+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:af2b834d:::obj153:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.664+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af2b834d:::obj153:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af2b834d:::obj153:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:af2dbe17:::obj414:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:af2dbe17:::obj414:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:af2dbe17:::obj414:head 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:af2dbe17:::obj414:head v 107'414 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.664+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af2dbe17:::obj414:head, version: 107'414, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af2dbe17:::obj414:head@107'414, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af2b834d:::obj153:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af2b834d:::obj153:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:af2b834d:::obj153:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:af2b834d:::obj153:head) v3 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:aecaa835:::obj157:head local-lis/les=112/113 n=411 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:af2b834d:::obj153:head) 2026-02-11T23:31:17.664+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af2b834d:::obj153:head) v3 finish latency 0.000010 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.664+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2a80 con 0x3e4a000 2026-02-11T23:31:17.664+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:af2dbe17:::obj414:head)]) -- 0x57dfe00 con 0x3e4a000 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.664+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 873 ==== MOSDPGPush(4.0 115/112 [PushOp(4:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.664+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.664+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.664+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 874 ==== pg_backfill(progress 4.0 e 115/115 lb 4:af2dbe17:::obj414:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.664+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:af2dbe17:::obj414:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.664+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af2dbe17:::obj414:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af2dbe17:::obj414:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:af44a1d6:::obj35:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:af44a1d6:::obj35:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:af44a1d6:::obj35:head 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:af44a1d6:::obj35:head v 107'35 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.664+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af44a1d6:::obj35:head, version: 107'35, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af44a1d6:::obj35:head@107'35, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af2dbe17:::obj414:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af2dbe17:::obj414:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:af2dbe17:::obj414:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:af2dbe17:::obj414:head) v3 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2b834d:::obj153:head local-lis/les=112/113 n=412 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:af2dbe17:::obj414:head) 2026-02-11T23:31:17.664+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.664+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af2dbe17:::obj414:head) v3 finish latency 0.000016 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.664+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.664+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.664+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0c00 con 0x3e4a000 2026-02-11T23:31:17.664+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:af44a1d6:::obj35:head)]) -- 0x4bf1080 con 0x3e4a000 2026-02-11T23:31:17.664+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.665+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 875 ==== MOSDPGPush(4.0 115/112 [PushOp(4:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.665+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.665+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.665+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 876 ==== pg_backfill(progress 4.0 e 115/115 lb 4:af44a1d6:::obj35:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5600 con 0x3e4a000 2026-02-11T23:31:17.665+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:af44a1d6:::obj35:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.665+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af44a1d6:::obj35:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af44a1d6:::obj35:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:af5f95cb:::obj12:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:af5f95cb:::obj12:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:af5f95cb:::obj12:head 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:af5f95cb:::obj12:head v 107'12 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.665+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af5f95cb:::obj12:head, version: 107'12, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af5f95cb:::obj12:head@107'12, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af44a1d6:::obj35:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af44a1d6:::obj35:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:af44a1d6:::obj35:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:af44a1d6:::obj35:head) v3 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af2dbe17:::obj414:head local-lis/les=112/113 n=413 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:af44a1d6:::obj35:head) 2026-02-11T23:31:17.665+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af44a1d6:::obj35:head) v3 finish latency 0.000012 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.665+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2780 con 0x3e4a000 2026-02-11T23:31:17.665+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:af5f95cb:::obj12:head)]) -- 0x57f0180 con 0x3e4a000 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.665+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 877 ==== MOSDPGPush(4.0 115/112 [PushOp(4:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.665+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.665+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.665+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 878 ==== pg_backfill(progress 4.0 e 115/115 lb 4:af5f95cb:::obj12:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.665+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:af5f95cb:::obj12:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.665+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af5f95cb:::obj12:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af5f95cb:::obj12:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:af93a04b:::obj437:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:af93a04b:::obj437:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:af93a04b:::obj437:head 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:af93a04b:::obj437:head v 107'437 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.665+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.665+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af93a04b:::obj437:head, version: 107'437, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af93a04b:::obj437:head@107'437, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.665+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af5f95cb:::obj12:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af5f95cb:::obj12:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 115/115 lb 4:af5f95cb:::obj12:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:af5f95cb:::obj12:head) v3 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af44a1d6:::obj35:head local-lis/les=112/113 n=414 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:af5f95cb:::obj12:head) 2026-02-11T23:31:17.665+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.665+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af5f95cb:::obj12:head) v3 finish latency 0.000015 2026-02-11T23:31:17.665+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.666+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.666+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.666+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0d80 con 0x3e4a000 2026-02-11T23:31:17.666+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:af93a04b:::obj437:head)]) -- 0x4ba2d80 con 0x3e4a000 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.666+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 879 ==== MOSDPGPush(4.0 115/112 [PushOp(4:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.666+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.666+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.666+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 880 ==== pg_backfill(progress 4.0 e 115/115 lb 4:af93a04b:::obj437:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.666+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:af93a04b:::obj437:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.666+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.666+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af93a04b:::obj437:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.666+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.666+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.666+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.666+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.666+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.666+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af93a04b:::obj437:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.666+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.666+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:af9ff462:::obj165:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.666+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:af9ff462:::obj165:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.666+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:af9ff462:::obj165:head 2026-02-11T23:31:17.666+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:af9ff462:::obj165:head v 107'165 2026-02-11T23:31:17.666+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.666+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.666+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:af9ff462:::obj165:head, version: 107'165, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:af9ff462:::obj165:head@107'165, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.666+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af93a04b:::obj437:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.666+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af93a04b:::obj437:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 115/115 lb 4:af93a04b:::obj437:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.666+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.666+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:af93a04b:::obj437:head) v3 2026-02-11T23:31:17.666+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af5f95cb:::obj12:head local-lis/les=112/113 n=415 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:af93a04b:::obj437:head) 2026-02-11T23:31:17.666+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.666+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af93a04b:::obj437:head) v3 finish latency 0.000011 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.666+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.666+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.666+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2180 con 0x3e4a000 2026-02-11T23:31:17.666+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:af9ff462:::obj165:head)]) -- 0x57f0300 con 0x3e4a000 2026-02-11T23:31:17.666+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.666+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 881 ==== MOSDPGPush(4.0 115/112 [PushOp(4:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3e4a000 2026-02-11T23:31:17.666+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.666+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.667+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 882 ==== pg_backfill(progress 4.0 e 115/115 lb 4:af9ff462:::obj165:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.667+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:af9ff462:::obj165:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.667+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af9ff462:::obj165:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af9ff462:::obj165:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:afacbcad:::obj315:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:afacbcad:::obj315:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:afacbcad:::obj315:head 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:afacbcad:::obj315:head v 107'315 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.667+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:afacbcad:::obj315:head, version: 107'315, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afacbcad:::obj315:head@107'315, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:af9ff462:::obj165:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af9ff462:::obj165:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:af9ff462:::obj165:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:af9ff462:::obj165:head) v3 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af93a04b:::obj437:head local-lis/les=112/113 n=416 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:af9ff462:::obj165:head) 2026-02-11T23:31:17.667+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:af9ff462:::obj165:head) v3 finish latency 0.000017 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.667+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2480 con 0x3e4a000 2026-02-11T23:31:17.667+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:afacbcad:::obj315:head)]) -- 0x4bf0f00 con 0x3e4a000 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.667+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 883 ==== MOSDPGPush(4.0 115/112 [PushOp(4:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43ea1a0 con 0x3e4a000 2026-02-11T23:31:17.667+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.667+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.667+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 884 ==== pg_backfill(progress 4.0 e 115/115 lb 4:afacbcad:::obj315:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.667+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:afacbcad:::obj315:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.667+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:afacbcad:::obj315:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:afacbcad:::obj315:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:afbdc8ea:::obj55:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:afbdc8ea:::obj55:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:afbdc8ea:::obj55:head 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:afbdc8ea:::obj55:head v 107'55 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.667+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.667+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:afbdc8ea:::obj55:head, version: 107'55, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afbdc8ea:::obj55:head@107'55, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:afacbcad:::obj315:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:afacbcad:::obj315:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:afacbcad:::obj315:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:afacbcad:::obj315:head) v3 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:af9ff462:::obj165:head local-lis/les=112/113 n=417 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:afacbcad:::obj315:head) 2026-02-11T23:31:17.667+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:afacbcad:::obj315:head) v3 finish latency 0.000012 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.667+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.667+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.667+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2300 con 0x3e4a000 2026-02-11T23:31:17.667+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:afbdc8ea:::obj55:head)]) -- 0x57f0600 con 0x3e4a000 2026-02-11T23:31:17.667+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.668+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 885 ==== MOSDPGPush(4.0 115/112 [PushOp(4:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3e4a000 2026-02-11T23:31:17.668+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.668+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.668+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 886 ==== pg_backfill(progress 4.0 e 115/115 lb 4:afbdc8ea:::obj55:head) ==== 1061+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.668+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:afbdc8ea:::obj55:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.668+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:afbdc8ea:::obj55:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:afbdc8ea:::obj55:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:afd348d8:::obj184:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:afd348d8:::obj184:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:afd348d8:::obj184:head 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:afd348d8:::obj184:head v 107'184 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.668+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:afd348d8:::obj184:head, version: 107'184, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:afd348d8:::obj184:head@107'184, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:afbdc8ea:::obj55:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:afbdc8ea:::obj55:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:afbdc8ea:::obj55:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:afbdc8ea:::obj55:head) v3 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afacbcad:::obj315:head local-lis/les=112/113 n=418 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:afbdc8ea:::obj55:head) 2026-02-11T23:31:17.668+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:afbdc8ea:::obj55:head) v3 finish latency 0.000010 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.668+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4be9500 con 0x3e4a000 2026-02-11T23:31:17.668+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:afd348d8:::obj184:head)]) -- 0x57f0780 con 0x3e4a000 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.668+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 887 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dd40 con 0x3e4a000 2026-02-11T23:31:17.668+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.668+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.668+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 888 ==== pg_backfill(progress 4.0 e 115/115 lb 4:afd348d8:::obj184:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.668+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:afd348d8:::obj184:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.668+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:afd348d8:::obj184:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:afd348d8:::obj184:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b0395c4b:::obj158:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b0395c4b:::obj158:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b0395c4b:::obj158:head 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b0395c4b:::obj158:head v 107'158 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.668+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b0395c4b:::obj158:head, version: 107'158, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b0395c4b:::obj158:head@107'158, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:afd348d8:::obj184:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:afd348d8:::obj184:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:afd348d8:::obj184:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:afd348d8:::obj184:head) v3 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afbdc8ea:::obj55:head local-lis/les=112/113 n=419 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:afd348d8:::obj184:head) 2026-02-11T23:31:17.668+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.668+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:afd348d8:::obj184:head) v3 finish latency 0.000016 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.668+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.668+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.668+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.668+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b0395c4b:::obj158:head)]) -- 0x53b2a80 con 0x3e4a000 2026-02-11T23:31:17.668+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.669+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 889 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183380 con 0x3e4a000 2026-02-11T23:31:17.669+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.669+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.669+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.669+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 890 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b0395c4b:::obj158:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.669+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b0395c4b:::obj158:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.669+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.669+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b0395c4b:::obj158:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.669+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.669+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.669+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.669+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b0395c4b:::obj158:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.669+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.669+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.669+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.669+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.669+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b133d16d:::obj222:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.669+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b133d16d:::obj222:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.669+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b133d16d:::obj222:head 2026-02-11T23:31:17.669+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b133d16d:::obj222:head v 107'222 2026-02-11T23:31:17.669+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.669+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.669+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b133d16d:::obj222:head, version: 107'222, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b133d16d:::obj222:head@107'222, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.669+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.669+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.669+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b0395c4b:::obj158:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.669+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b0395c4b:::obj158:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:b0395c4b:::obj158:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.669+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.669+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b0395c4b:::obj158:head) v3 2026-02-11T23:31:17.669+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:afd348d8:::obj184:head local-lis/les=112/113 n=420 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b0395c4b:::obj158:head) 2026-02-11T23:31:17.669+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.669+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b0395c4b:::obj158:head) v3 finish latency 0.000011 2026-02-11T23:31:17.669+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.669+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.669+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.669+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.669+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.669+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.669+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.669+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf1200 con 0x3e4a000 2026-02-11T23:31:17.669+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b133d16d:::obj222:head)]) -- 0x57f0a80 con 0x3e4a000 2026-02-11T23:31:17.669+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.670+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 891 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de44e0 con 0x3e4a000 2026-02-11T23:31:17.670+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.670+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.670+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 892 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b133d16d:::obj222:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3e4a000 2026-02-11T23:31:17.670+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b133d16d:::obj222:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.670+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b133d16d:::obj222:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b133d16d:::obj222:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b164ceff:::obj421:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b164ceff:::obj421:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b164ceff:::obj421:head 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b164ceff:::obj421:head v 107'421 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.670+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b164ceff:::obj421:head, version: 107'421, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b164ceff:::obj421:head@107'421, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b133d16d:::obj222:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b133d16d:::obj222:head) v3 prio 15 cost 0 latency 0.000082 pg_backfill(progress 4.0 e 115/115 lb 4:b133d16d:::obj222:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b133d16d:::obj222:head) v3 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b0395c4b:::obj158:head local-lis/les=112/113 n=421 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b133d16d:::obj222:head) 2026-02-11T23:31:17.670+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b133d16d:::obj222:head) v3 finish latency 0.000014 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.670+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.670+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b164ceff:::obj421:head)]) -- 0x4bf0a80 con 0x3e4a000 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.670+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 893 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d040 con 0x3e4a000 2026-02-11T23:31:17.670+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.670+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.670+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 894 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b164ceff:::obj421:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.670+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b164ceff:::obj421:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.670+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b164ceff:::obj421:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b164ceff:::obj421:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b1721c5a:::obj526:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b1721c5a:::obj526:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b1721c5a:::obj526:head 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b1721c5a:::obj526:head v 107'526 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.670+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.670+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b1721c5a:::obj526:head, version: 107'526, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1721c5a:::obj526:head@107'526, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b164ceff:::obj421:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b164ceff:::obj421:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:b164ceff:::obj421:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b164ceff:::obj421:head) v3 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b133d16d:::obj222:head local-lis/les=112/113 n=422 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b164ceff:::obj421:head) 2026-02-11T23:31:17.670+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b164ceff:::obj421:head) v3 finish latency 0.000013 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.670+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.670+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.670+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57df500 con 0x3e4a000 2026-02-11T23:31:17.670+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b1721c5a:::obj526:head)]) -- 0x57f0d80 con 0x3e4a000 2026-02-11T23:31:17.670+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.671+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 895 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.671+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.671+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.671+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 896 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b1721c5a:::obj526:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3e4a000 2026-02-11T23:31:17.671+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b1721c5a:::obj526:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.671+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b1721c5a:::obj526:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b1721c5a:::obj526:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b1ca68cf:::obj358:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b1ca68cf:::obj358:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b1ca68cf:::obj358:head 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b1ca68cf:::obj358:head v 107'358 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.671+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b1ca68cf:::obj358:head, version: 107'358, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1ca68cf:::obj358:head@107'358, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b1721c5a:::obj526:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b1721c5a:::obj526:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:b1721c5a:::obj526:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b1721c5a:::obj526:head) v3 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b164ceff:::obj421:head local-lis/les=112/113 n=423 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b1721c5a:::obj526:head) 2026-02-11T23:31:17.671+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b1721c5a:::obj526:head) v3 finish latency 0.000015 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.671+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2f00 con 0x3e4a000 2026-02-11T23:31:17.671+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b1ca68cf:::obj358:head)]) -- 0x4bf0900 con 0x3e4a000 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.671+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 897 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.671+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.671+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.671+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 898 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b1ca68cf:::obj358:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bc100 con 0x3e4a000 2026-02-11T23:31:17.671+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b1ca68cf:::obj358:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.671+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b1ca68cf:::obj358:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b1ca68cf:::obj358:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b1e2ea00:::obj289:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b1e2ea00:::obj289:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b1e2ea00:::obj289:head 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b1e2ea00:::obj289:head v 107'289 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.671+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.671+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b1e2ea00:::obj289:head, version: 107'289, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b1e2ea00:::obj289:head@107'289, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.671+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b1ca68cf:::obj358:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b1ca68cf:::obj358:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:b1ca68cf:::obj358:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b1ca68cf:::obj358:head) v3 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1721c5a:::obj526:head local-lis/les=112/113 n=424 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b1ca68cf:::obj358:head) 2026-02-11T23:31:17.671+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.671+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b1ca68cf:::obj358:head) v3 finish latency 0.000012 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.671+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.672+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.672+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4180 con 0x3e4a000 2026-02-11T23:31:17.672+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b1e2ea00:::obj289:head)]) -- 0x57f1080 con 0x3e4a000 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.672+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 899 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.672+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.672+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.672+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 900 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b1e2ea00:::obj289:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3e4a000 2026-02-11T23:31:17.672+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b1e2ea00:::obj289:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.672+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b1e2ea00:::obj289:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.672+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.672+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b1e2ea00:::obj289:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.672+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.672+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.672+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b26ae718:::obj317:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b26ae718:::obj317:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.672+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b26ae718:::obj317:head 2026-02-11T23:31:17.672+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b26ae718:::obj317:head v 107'317 2026-02-11T23:31:17.672+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.672+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.672+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b26ae718:::obj317:head, version: 107'317, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b26ae718:::obj317:head@107'317, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b1e2ea00:::obj289:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b1e2ea00:::obj289:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 115/115 lb 4:b1e2ea00:::obj289:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b1e2ea00:::obj289:head) v3 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1ca68cf:::obj358:head local-lis/les=112/113 n=425 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b1e2ea00:::obj289:head) 2026-02-11T23:31:17.672+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b1e2ea00:::obj289:head) v3 finish latency 0.000013 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.672+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.672+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3200 con 0x3e4a000 2026-02-11T23:31:17.672+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b26ae718:::obj317:head)]) -- 0x4ae4600 con 0x3e4a000 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.672+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 901 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.672+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.672+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.672+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 902 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b26ae718:::obj317:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3e4a000 2026-02-11T23:31:17.672+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b26ae718:::obj317:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.672+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b26ae718:::obj317:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.672+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b26ae718:::obj317:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b2b26856:::obj586:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.672+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b2b26856:::obj586:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b2b26856:::obj586:head 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b2b26856:::obj586:head v 107'586 2026-02-11T23:31:17.672+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.673+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.673+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b2b26856:::obj586:head, version: 107'586, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2b26856:::obj586:head@107'586, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.673+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b26ae718:::obj317:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b26ae718:::obj317:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:b26ae718:::obj317:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b26ae718:::obj317:head) v3 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b1e2ea00:::obj289:head local-lis/les=112/113 n=426 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b26ae718:::obj317:head) 2026-02-11T23:31:17.673+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b26ae718:::obj317:head) v3 finish latency 0.000012 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.673+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.673+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3680 con 0x3e4a000 2026-02-11T23:31:17.673+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b2b26856:::obj586:head)]) -- 0x57f1380 con 0x3e4a000 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.673+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 903 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ebba0 con 0x3e4a000 2026-02-11T23:31:17.673+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.673+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.673+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 904 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b2b26856:::obj586:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3e4a000 2026-02-11T23:31:17.673+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b2b26856:::obj586:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.673+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b2b26856:::obj586:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.673+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.673+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b2b26856:::obj586:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b2ec4718:::obj500:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b2ec4718:::obj500:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b2ec4718:::obj500:head 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b2ec4718:::obj500:head v 107'500 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.673+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b2ec4718:::obj500:head, version: 107'500, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b2ec4718:::obj500:head@107'500, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.673+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.673+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b2b26856:::obj586:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.673+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b2b26856:::obj586:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:b2b26856:::obj586:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.673+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.673+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b2b26856:::obj586:head) v3 2026-02-11T23:31:17.673+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b26ae718:::obj317:head local-lis/les=112/113 n=427 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b2b26856:::obj586:head) 2026-02-11T23:31:17.673+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.673+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b2b26856:::obj586:head) v3 finish latency 0.000016 2026-02-11T23:31:17.673+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.673+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.673+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.673+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57de180 con 0x3e4a000 2026-02-11T23:31:17.673+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b2ec4718:::obj500:head)]) -- 0x4f50180 con 0x3e4a000 2026-02-11T23:31:17.673+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.674+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 905 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3e4a000 2026-02-11T23:31:17.674+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.674+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.674+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 906 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b2ec4718:::obj500:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3e4a000 2026-02-11T23:31:17.674+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b2ec4718:::obj500:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.674+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b2ec4718:::obj500:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b2ec4718:::obj500:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b33086a8:::obj36:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b33086a8:::obj36:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b33086a8:::obj36:head 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b33086a8:::obj36:head v 107'36 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.674+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b33086a8:::obj36:head, version: 107'36, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b33086a8:::obj36:head@107'36, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b2ec4718:::obj500:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b2ec4718:::obj500:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 115/115 lb 4:b2ec4718:::obj500:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b2ec4718:::obj500:head) v3 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2b26856:::obj586:head local-lis/les=112/113 n=428 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b2ec4718:::obj500:head) 2026-02-11T23:31:17.674+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b2ec4718:::obj500:head) v3 finish latency 0.000012 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.674+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57df080 con 0x3e4a000 2026-02-11T23:31:17.674+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b33086a8:::obj36:head)]) -- 0x57f1680 con 0x3e4a000 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.674+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 907 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3e4a000 2026-02-11T23:31:17.674+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.674+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.674+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 908 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b33086a8:::obj36:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3e4a000 2026-02-11T23:31:17.674+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b33086a8:::obj36:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.674+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b33086a8:::obj36:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b33086a8:::obj36:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b4726009:::obj503:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b4726009:::obj503:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b4726009:::obj503:head 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b4726009:::obj503:head v 107'503 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.674+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b4726009:::obj503:head, version: 107'503, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b4726009:::obj503:head@107'503, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b33086a8:::obj36:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b33086a8:::obj36:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:b33086a8:::obj36:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b33086a8:::obj36:head) v3 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b2ec4718:::obj500:head local-lis/les=112/113 n=429 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b33086a8:::obj36:head) 2026-02-11T23:31:17.674+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.674+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b33086a8:::obj36:head) v3 finish latency 0.000016 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.674+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.674+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.674+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.674+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b4726009:::obj503:head)]) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.674+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.675+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 909 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3e4a000 2026-02-11T23:31:17.675+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.675+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.675+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 910 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b4726009:::obj503:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3e4a000 2026-02-11T23:31:17.675+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b4726009:::obj503:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.675+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b4726009:::obj503:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b4726009:::obj503:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b5298834:::obj386:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b5298834:::obj386:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b5298834:::obj386:head 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b5298834:::obj386:head v 107'386 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.675+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5298834:::obj386:head, version: 107'386, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5298834:::obj386:head@107'386, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b4726009:::obj503:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.675+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b4726009:::obj503:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:b4726009:::obj503:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.675+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.675+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b4726009:::obj503:head) v3 2026-02-11T23:31:17.675+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b33086a8:::obj36:head local-lis/les=112/113 n=430 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b4726009:::obj503:head) 2026-02-11T23:31:17.675+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.675+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b4726009:::obj503:head) v3 finish latency 0.000011 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.675+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.675+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57ded80 con 0x3e4a000 2026-02-11T23:31:17.675+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b5298834:::obj386:head)]) -- 0x57f1800 con 0x3e4a000 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.675+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 911 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3e4a000 2026-02-11T23:31:17.675+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.675+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.675+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 912 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b5298834:::obj386:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3e4a000 2026-02-11T23:31:17.675+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5298834:::obj386:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.675+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5298834:::obj386:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5298834:::obj386:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b5339e65:::obj355:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b5339e65:::obj355:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b5339e65:::obj355:head 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b5339e65:::obj355:head v 107'355 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.675+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.675+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5339e65:::obj355:head, version: 107'355, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5339e65:::obj355:head@107'355, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.675+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5298834:::obj386:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.675+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5298834:::obj386:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:b5298834:::obj386:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.675+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.675+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b5298834:::obj386:head) v3 2026-02-11T23:31:17.675+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b4726009:::obj503:head local-lis/les=112/113 n=431 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b5298834:::obj386:head) 2026-02-11T23:31:17.675+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.675+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5298834:::obj386:head) v3 finish latency 0.000011 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.675+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.676+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.676+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.676+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.676+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b5339e65:::obj355:head)]) -- 0x57f1b00 con 0x3e4a000 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.676+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 913 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2340 con 0x3e4a000 2026-02-11T23:31:17.676+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.676+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.676+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.676+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 914 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b5339e65:::obj355:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3e4a000 2026-02-11T23:31:17.676+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5339e65:::obj355:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.676+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.676+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5339e65:::obj355:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.676+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.676+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5339e65:::obj355:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.676+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.676+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.676+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.676+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.676+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b5445b0d:::obj455:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.676+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b5445b0d:::obj455:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.676+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b5445b0d:::obj455:head 2026-02-11T23:31:17.676+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b5445b0d:::obj455:head v 107'455 2026-02-11T23:31:17.676+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.676+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.676+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5445b0d:::obj455:head, version: 107'455, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5445b0d:::obj455:head@107'455, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.676+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5339e65:::obj355:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.676+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5339e65:::obj355:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:b5339e65:::obj355:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.676+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.676+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b5339e65:::obj355:head) v3 2026-02-11T23:31:17.676+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5298834:::obj386:head local-lis/les=112/113 n=432 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b5339e65:::obj355:head) 2026-02-11T23:31:17.676+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.676+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5339e65:::obj355:head) v3 finish latency 0.000010 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.676+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.676+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.676+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3c80 con 0x3e4a000 2026-02-11T23:31:17.676+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b5445b0d:::obj455:head)]) -- 0x57f1e00 con 0x3e4a000 2026-02-11T23:31:17.676+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.677+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 915 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.677+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.677+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.677+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 916 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b5445b0d:::obj455:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3e4a000 2026-02-11T23:31:17.677+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5445b0d:::obj455:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.677+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5445b0d:::obj455:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5445b0d:::obj455:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b59cea89:::obj183:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b59cea89:::obj183:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b59cea89:::obj183:head 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b59cea89:::obj183:head v 107'183 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.677+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b59cea89:::obj183:head, version: 107'183, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b59cea89:::obj183:head@107'183, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5445b0d:::obj455:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.677+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5445b0d:::obj455:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:b5445b0d:::obj455:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.677+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.677+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b5445b0d:::obj455:head) v3 2026-02-11T23:31:17.677+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5339e65:::obj355:head local-lis/les=112/113 n=433 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b5445b0d:::obj455:head) 2026-02-11T23:31:17.677+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.677+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5445b0d:::obj455:head) v3 finish latency 0.000016 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.677+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c2de00 con 0x3e4a000 2026-02-11T23:31:17.677+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b59cea89:::obj183:head)]) -- 0x53b3b00 con 0x3e4a000 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.677+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 917 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.677+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.677+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.677+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 918 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b59cea89:::obj183:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3e4a000 2026-02-11T23:31:17.677+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b59cea89:::obj183:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.677+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b59cea89:::obj183:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b59cea89:::obj183:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b5a67629:::obj219:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b5a67629:::obj219:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b5a67629:::obj219:head 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b5a67629:::obj219:head v 107'219 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.677+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b5a67629:::obj219:head, version: 107'219, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5a67629:::obj219:head@107'219, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b59cea89:::obj183:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.677+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b59cea89:::obj183:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:b59cea89:::obj183:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.677+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.677+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b59cea89:::obj183:head) v3 2026-02-11T23:31:17.677+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5445b0d:::obj455:head local-lis/les=112/113 n=434 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b59cea89:::obj183:head) 2026-02-11T23:31:17.677+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.677+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b59cea89:::obj183:head) v3 finish latency 0.000018 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.677+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.677+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.677+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57de780 con 0x3e4a000 2026-02-11T23:31:17.677+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b5a67629:::obj219:head)]) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.677+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.678+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 919 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.678+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.678+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.678+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 920 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b5a67629:::obj219:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5bb80 con 0x3e4a000 2026-02-11T23:31:17.678+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5a67629:::obj219:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.678+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5a67629:::obj219:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5a67629:::obj219:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b609c5a3:::obj261:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b609c5a3:::obj261:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b609c5a3:::obj261:head 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b609c5a3:::obj261:head v 107'261 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.678+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b609c5a3:::obj261:head, version: 107'261, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b609c5a3:::obj261:head@107'261, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b5a67629:::obj219:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.678+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5a67629:::obj219:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:b5a67629:::obj219:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.678+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.678+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b5a67629:::obj219:head) v3 2026-02-11T23:31:17.678+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b59cea89:::obj183:head local-lis/les=112/113 n=435 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b5a67629:::obj219:head) 2026-02-11T23:31:17.678+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.678+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b5a67629:::obj219:head) v3 finish latency 0.000011 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.678+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.678+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55de000 con 0x3e4a000 2026-02-11T23:31:17.678+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b609c5a3:::obj261:head)]) -- 0x57f1380 con 0x3e4a000 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.678+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 921 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.678+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.678+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.678+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 922 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b609c5a3:::obj261:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3e4a000 2026-02-11T23:31:17.678+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b609c5a3:::obj261:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.678+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b609c5a3:::obj261:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b609c5a3:::obj261:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b66eab34:::obj112:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b66eab34:::obj112:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b66eab34:::obj112:head 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b66eab34:::obj112:head v 107'112 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.678+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.678+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b66eab34:::obj112:head, version: 107'112, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b66eab34:::obj112:head@107'112, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.678+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b609c5a3:::obj261:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.678+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b609c5a3:::obj261:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:b609c5a3:::obj261:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.678+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.678+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b609c5a3:::obj261:head) v3 2026-02-11T23:31:17.678+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b5a67629:::obj219:head local-lis/les=112/113 n=436 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b609c5a3:::obj261:head) 2026-02-11T23:31:17.678+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.678+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b609c5a3:::obj261:head) v3 finish latency 0.000010 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.678+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.679+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.679+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.679+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55de180 con 0x3e4a000 2026-02-11T23:31:17.679+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b66eab34:::obj112:head)]) -- 0x4ae4600 con 0x3e4a000 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.679+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 923 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d040 con 0x3e4a000 2026-02-11T23:31:17.679+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.679+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.679+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 924 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b66eab34:::obj112:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3e4a000 2026-02-11T23:31:17.679+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b66eab34:::obj112:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.679+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b66eab34:::obj112:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.679+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.679+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.679+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.679+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.679+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b66eab34:::obj112:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.679+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.679+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b6a90992:::obj552:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b6a90992:::obj552:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.679+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b6a90992:::obj552:head 2026-02-11T23:31:17.679+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b6a90992:::obj552:head v 107'552 2026-02-11T23:31:17.679+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.679+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.679+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b6a90992:::obj552:head, version: 107'552, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6a90992:::obj552:head@107'552, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.679+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.679+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b66eab34:::obj112:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.679+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b66eab34:::obj112:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 115/115 lb 4:b66eab34:::obj112:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.679+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.679+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b66eab34:::obj112:head) v3 2026-02-11T23:31:17.679+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b609c5a3:::obj261:head local-lis/les=112/113 n=437 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b66eab34:::obj112:head) 2026-02-11T23:31:17.679+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.679+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b66eab34:::obj112:head) v3 finish latency 0.000015 2026-02-11T23:31:17.679+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.679+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.679+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.679+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55de600 con 0x3e4a000 2026-02-11T23:31:17.679+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b6a90992:::obj552:head)]) -- 0x55de300 con 0x3e4a000 2026-02-11T23:31:17.679+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.679+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 925 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de44e0 con 0x3e4a000 2026-02-11T23:31:17.679+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.679+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.679+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.679+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 926 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b6a90992:::obj552:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3e4a000 2026-02-11T23:31:17.679+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b6a90992:::obj552:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.679+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b6a90992:::obj552:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.679+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.679+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.679+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b6a90992:::obj552:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b6dfbaf7:::obj426:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.680+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b6dfbaf7:::obj426:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b6dfbaf7:::obj426:head 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b6dfbaf7:::obj426:head v 107'426 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.680+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b6dfbaf7:::obj426:head, version: 107'426, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b6dfbaf7:::obj426:head@107'426, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.680+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b6a90992:::obj552:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b6a90992:::obj552:head) v3 prio 15 cost 0 latency 0.000081 pg_backfill(progress 4.0 e 115/115 lb 4:b6a90992:::obj552:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b6a90992:::obj552:head) v3 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b66eab34:::obj112:head local-lis/les=112/113 n=438 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b6a90992:::obj552:head) 2026-02-11T23:31:17.680+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b6a90992:::obj552:head) v3 finish latency 0.000012 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.680+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.680+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55de780 con 0x3e4a000 2026-02-11T23:31:17.680+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b6dfbaf7:::obj426:head)]) -- 0x57f1080 con 0x3e4a000 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.680+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 927 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183380 con 0x3e4a000 2026-02-11T23:31:17.680+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.680+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.680+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.680+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 928 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b6dfbaf7:::obj426:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3e4a000 2026-02-11T23:31:17.680+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b6dfbaf7:::obj426:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.680+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b6dfbaf7:::obj426:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.680+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b6dfbaf7:::obj426:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.680+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.680+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 115/112 [PushOp(4:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b70eecc2:::obj59:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.680+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b70eecc2:::obj59:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b70eecc2:::obj59:head 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b70eecc2:::obj59:head v 107'59 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.680+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.680+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b70eecc2:::obj59:head, version: 107'59, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b70eecc2:::obj59:head@107'59, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.680+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b6dfbaf7:::obj426:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b6dfbaf7:::obj426:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 115/115 lb 4:b6dfbaf7:::obj426:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b6dfbaf7:::obj426:head) v3 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6a90992:::obj552:head local-lis/les=112/113 n=439 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b6dfbaf7:::obj426:head) 2026-02-11T23:31:17.680+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b6dfbaf7:::obj426:head) v3 finish latency 0.000010 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.680+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.680+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.680+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55de900 con 0x3e4a000 2026-02-11T23:31:17.680+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b70eecc2:::obj59:head)]) -- 0x4bf0900 con 0x3e4a000 2026-02-11T23:31:17.680+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.681+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 929 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dd40 con 0x3e4a000 2026-02-11T23:31:17.681+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.681+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.681+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 930 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b70eecc2:::obj59:head) ==== 1061+0+0 (unknown 0 0 0) 0x5162000 con 0x3e4a000 2026-02-11T23:31:17.681+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b70eecc2:::obj59:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.681+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b70eecc2:::obj59:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b70eecc2:::obj59:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b75b9722:::obj495:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b75b9722:::obj495:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b75b9722:::obj495:head 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b75b9722:::obj495:head v 107'495 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.681+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b75b9722:::obj495:head, version: 107'495, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b75b9722:::obj495:head@107'495, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b70eecc2:::obj59:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b70eecc2:::obj59:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:b70eecc2:::obj59:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b70eecc2:::obj59:head) v3 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b6dfbaf7:::obj426:head local-lis/les=112/113 n=440 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b70eecc2:::obj59:head) 2026-02-11T23:31:17.681+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b70eecc2:::obj59:head) v3 finish latency 0.000018 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.681+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55ded80 con 0x3e4a000 2026-02-11T23:31:17.681+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b75b9722:::obj495:head)]) -- 0x55dea80 con 0x3e4a000 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.681+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 931 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3e4a000 2026-02-11T23:31:17.681+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.681+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.681+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 932 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b75b9722:::obj495:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162580 con 0x3e4a000 2026-02-11T23:31:17.681+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b75b9722:::obj495:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.681+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b75b9722:::obj495:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b75b9722:::obj495:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b841f58a:::obj361:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b841f58a:::obj361:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b841f58a:::obj361:head 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b841f58a:::obj361:head v 107'361 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.681+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.681+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b841f58a:::obj361:head, version: 107'361, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b841f58a:::obj361:head@107'361, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b75b9722:::obj495:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b75b9722:::obj495:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:b75b9722:::obj495:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b75b9722:::obj495:head) v3 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b70eecc2:::obj59:head local-lis/les=112/113 n=441 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b75b9722:::obj495:head) 2026-02-11T23:31:17.681+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b75b9722:::obj495:head) v3 finish latency 0.000012 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.681+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.681+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.681+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55def00 con 0x3e4a000 2026-02-11T23:31:17.681+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b841f58a:::obj361:head)]) -- 0x53b2f00 con 0x3e4a000 2026-02-11T23:31:17.681+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.682+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 933 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43ea1a0 con 0x3e4a000 2026-02-11T23:31:17.682+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.682+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.682+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 934 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b841f58a:::obj361:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3e4a000 2026-02-11T23:31:17.682+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b841f58a:::obj361:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.682+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b841f58a:::obj361:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b841f58a:::obj361:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b8510467:::obj40:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b8510467:::obj40:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b8510467:::obj40:head 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b8510467:::obj40:head v 107'40 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.682+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b8510467:::obj40:head, version: 107'40, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8510467:::obj40:head@107'40, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b841f58a:::obj361:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b841f58a:::obj361:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:b841f58a:::obj361:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b841f58a:::obj361:head) v3 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b75b9722:::obj495:head local-lis/les=112/113 n=442 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b841f58a:::obj361:head) 2026-02-11T23:31:17.682+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b841f58a:::obj361:head) v3 finish latency 0.000010 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.682+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55df080 con 0x3e4a000 2026-02-11T23:31:17.682+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b8510467:::obj40:head)]) -- 0x57df500 con 0x3e4a000 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.682+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 935 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3e4a000 2026-02-11T23:31:17.682+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.682+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.682+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 936 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b8510467:::obj40:head) ==== 1061+0+0 (unknown 0 0 0) 0x5163080 con 0x3e4a000 2026-02-11T23:31:17.682+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b8510467:::obj40:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.682+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b8510467:::obj40:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b8510467:::obj40:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b8f63b78:::obj291:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b8f63b78:::obj291:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b8f63b78:::obj291:head 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b8f63b78:::obj291:head v 107'291 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.682+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.682+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b8f63b78:::obj291:head, version: 107'291, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b8f63b78:::obj291:head@107'291, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.682+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b8510467:::obj40:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b8510467:::obj40:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:b8510467:::obj40:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b8510467:::obj40:head) v3 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b841f58a:::obj361:head local-lis/les=112/113 n=443 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b8510467:::obj40:head) 2026-02-11T23:31:17.682+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.682+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b8510467:::obj40:head) v3 finish latency 0.000016 2026-02-11T23:31:17.682+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.683+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.683+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.683+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55df500 con 0x3e4a000 2026-02-11T23:31:17.683+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b8f63b78:::obj291:head)]) -- 0x55df200 con 0x3e4a000 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.683+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 937 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.683+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.683+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.683+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 938 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b8f63b78:::obj291:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163600 con 0x3e4a000 2026-02-11T23:31:17.683+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b8f63b78:::obj291:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.683+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b8f63b78:::obj291:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.683+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.683+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.683+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.683+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b8f63b78:::obj291:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.683+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.683+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.683+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b92d6f18:::obj505:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b92d6f18:::obj505:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.683+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b92d6f18:::obj505:head 2026-02-11T23:31:17.683+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b92d6f18:::obj505:head v 107'505 2026-02-11T23:31:17.683+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.683+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.683+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b92d6f18:::obj505:head, version: 107'505, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b92d6f18:::obj505:head@107'505, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.683+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.683+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b8f63b78:::obj291:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.683+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b8f63b78:::obj291:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:b8f63b78:::obj291:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.683+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.683+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b8f63b78:::obj291:head) v3 2026-02-11T23:31:17.683+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8510467:::obj40:head local-lis/les=112/113 n=444 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b8f63b78:::obj291:head) 2026-02-11T23:31:17.683+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.683+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b8f63b78:::obj291:head) v3 finish latency 0.000015 2026-02-11T23:31:17.683+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.683+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.683+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.683+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55df980 con 0x3e4a000 2026-02-11T23:31:17.683+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b92d6f18:::obj505:head)]) -- 0x55df680 con 0x3e4a000 2026-02-11T23:31:17.683+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.684+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 939 ==== MOSDPGPush(4.0 115/112 [PushOp(4:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.684+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.684+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.684+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 940 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b92d6f18:::obj505:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163b80 con 0x3e4a000 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.684+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b92d6f18:::obj505:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.684+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b92d6f18:::obj505:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 115/112 [PushOp(4:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b92d6f18:::obj505:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:b9836a99:::obj18:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:b9836a99:::obj18:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:b9836a99:::obj18:head 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:b9836a99:::obj18:head v 107'18 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.684+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:b9836a99:::obj18:head, version: 107'18, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b9836a99:::obj18:head@107'18, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b92d6f18:::obj505:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b92d6f18:::obj505:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:b92d6f18:::obj505:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b92d6f18:::obj505:head) v3 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b8f63b78:::obj291:head local-lis/les=112/113 n=445 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b92d6f18:::obj505:head) 2026-02-11T23:31:17.684+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b92d6f18:::obj505:head) v3 finish latency 0.000012 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.684+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55dfb00 con 0x3e4a000 2026-02-11T23:31:17.684+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:b9836a99:::obj18:head)]) -- 0x4bf0a80 con 0x3e4a000 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.684+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 941 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.684+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.684+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.684+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 942 ==== pg_backfill(progress 4.0 e 115/115 lb 4:b9836a99:::obj18:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.684+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:b9836a99:::obj18:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.684+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b9836a99:::obj18:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b9836a99:::obj18:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ba083999:::obj295:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ba083999:::obj295:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ba083999:::obj295:head 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ba083999:::obj295:head v 107'295 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.684+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ba083999:::obj295:head, version: 107'295, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba083999:::obj295:head@107'295, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:b9836a99:::obj18:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b9836a99:::obj18:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:b9836a99:::obj18:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:b9836a99:::obj18:head) v3 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b92d6f18:::obj505:head local-lis/les=112/113 n=446 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:b9836a99:::obj18:head) 2026-02-11T23:31:17.684+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.684+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:b9836a99:::obj18:head) v3 finish latency 0.000016 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.684+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.684+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.684+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55df500 con 0x3e4a000 2026-02-11T23:31:17.684+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ba083999:::obj295:head)]) -- 0x55dfc80 con 0x3e4a000 2026-02-11T23:31:17.684+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.685+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 943 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.685+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.685+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.685+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 944 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ba083999:::obj295:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.685+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ba083999:::obj295:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.685+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ba083999:::obj295:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ba083999:::obj295:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ba3d236d:::obj529:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ba3d236d:::obj529:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ba3d236d:::obj529:head 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ba3d236d:::obj529:head v 107'529 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.685+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ba3d236d:::obj529:head, version: 107'529, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba3d236d:::obj529:head@107'529, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ba083999:::obj295:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.685+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ba083999:::obj295:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:ba083999:::obj295:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.685+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.685+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ba083999:::obj295:head) v3 2026-02-11T23:31:17.685+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:b9836a99:::obj18:head local-lis/les=112/113 n=447 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ba083999:::obj295:head) 2026-02-11T23:31:17.685+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.685+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ba083999:::obj295:head) v3 finish latency 0.000012 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.685+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.685+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57df500 con 0x3e4a000 2026-02-11T23:31:17.685+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ba3d236d:::obj529:head)]) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.685+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 945 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a21a0 con 0x3e4a000 2026-02-11T23:31:17.685+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.685+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.685+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 946 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ba3d236d:::obj529:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.685+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ba3d236d:::obj529:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.685+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ba3d236d:::obj529:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ba3d236d:::obj529:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ba98db39:::obj542:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ba98db39:::obj542:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ba98db39:::obj542:head 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ba98db39:::obj542:head v 107'542 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.685+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.685+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ba98db39:::obj542:head, version: 107'542, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ba98db39:::obj542:head@107'542, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.685+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ba3d236d:::obj529:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.685+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ba3d236d:::obj529:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:ba3d236d:::obj529:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.685+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.685+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ba3d236d:::obj529:head) v3 2026-02-11T23:31:17.685+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba083999:::obj295:head local-lis/les=112/113 n=448 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ba3d236d:::obj529:head) 2026-02-11T23:31:17.685+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.685+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ba3d236d:::obj529:head) v3 finish latency 0.000010 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.685+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.686+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55df080 con 0x3e4a000 2026-02-11T23:31:17.686+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ba98db39:::obj542:head)]) -- 0x4bf1200 con 0x3e4a000 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.686+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 947 ==== MOSDPGPush(4.0 115/112 [PushOp(4:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2680 con 0x3e4a000 2026-02-11T23:31:17.686+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.686+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.686+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 948 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ba98db39:::obj542:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.686+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ba98db39:::obj542:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.686+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ba98db39:::obj542:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ba98db39:::obj542:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:bb038e88:::obj54:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:bb038e88:::obj54:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:bb038e88:::obj54:head 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:bb038e88:::obj54:head v 107'54 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.686+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bb038e88:::obj54:head, version: 107'54, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bb038e88:::obj54:head@107'54, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ba98db39:::obj542:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ba98db39:::obj542:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:ba98db39:::obj542:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ba98db39:::obj542:head) v3 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba3d236d:::obj529:head local-lis/les=112/113 n=449 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ba98db39:::obj542:head) 2026-02-11T23:31:17.686+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ba98db39:::obj542:head) v3 finish latency 0.000010 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.686+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b2f00 con 0x3e4a000 2026-02-11T23:31:17.686+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:bb038e88:::obj54:head)]) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.686+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 949 ==== MOSDPGPush(4.0 115/112 [PushOp(4:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2b60 con 0x3e4a000 2026-02-11T23:31:17.686+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.686+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.686+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 950 ==== pg_backfill(progress 4.0 e 115/115 lb 4:bb038e88:::obj54:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.686+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:bb038e88:::obj54:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.686+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bb038e88:::obj54:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bb038e88:::obj54:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:bbf67374:::obj187:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:bbf67374:::obj187:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:bbf67374:::obj187:head 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:bbf67374:::obj187:head v 107'187 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.686+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.686+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bbf67374:::obj187:head, version: 107'187, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bbf67374:::obj187:head@107'187, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.686+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bb038e88:::obj54:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bb038e88:::obj54:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:bb038e88:::obj54:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:bb038e88:::obj54:head) v3 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ba98db39:::obj542:head local-lis/les=112/113 n=450 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:bb038e88:::obj54:head) 2026-02-11T23:31:17.686+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.686+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bb038e88:::obj54:head) v3 finish latency 0.000010 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.686+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.687+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.687+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55def00 con 0x3e4a000 2026-02-11T23:31:17.687+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:bbf67374:::obj187:head)]) -- 0x57f0780 con 0x3e4a000 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.687+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 951 ==== MOSDPGPush(4.0 115/112 [PushOp(4:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dfe1a0 con 0x3e4a000 2026-02-11T23:31:17.687+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.687+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.687+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 952 ==== pg_backfill(progress 4.0 e 115/115 lb 4:bbf67374:::obj187:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.687+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:bbf67374:::obj187:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.687+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bbf67374:::obj187:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bbf67374:::obj187:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.687+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.687+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.687+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.687+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.687+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:bc55eba4:::obj534:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:bc55eba4:::obj534:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.687+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:bc55eba4:::obj534:head 2026-02-11T23:31:17.687+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:bc55eba4:::obj534:head v 107'534 2026-02-11T23:31:17.687+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.687+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.687+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bc55eba4:::obj534:head, version: 107'534, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bc55eba4:::obj534:head@107'534, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bbf67374:::obj187:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bbf67374:::obj187:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:bbf67374:::obj187:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:bbf67374:::obj187:head) v3 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bb038e88:::obj54:head local-lis/les=112/113 n=451 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:bbf67374:::obj187:head) 2026-02-11T23:31:17.687+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bbf67374:::obj187:head) v3 finish latency 0.000015 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.687+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.687+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0900 con 0x3e4a000 2026-02-11T23:31:17.687+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:bc55eba4:::obj534:head)]) -- 0x55dea80 con 0x3e4a000 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.687+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 953 ==== MOSDPGPush(4.0 115/112 [PushOp(4:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3520 con 0x3e4a000 2026-02-11T23:31:17.687+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.687+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.687+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 954 ==== pg_backfill(progress 4.0 e 115/115 lb 4:bc55eba4:::obj534:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.687+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:bc55eba4:::obj534:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.687+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bc55eba4:::obj534:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.687+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bc55eba4:::obj534:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:bd84d997:::obj292:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.687+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:bd84d997:::obj292:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:bd84d997:::obj292:head 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:bd84d997:::obj292:head v 107'292 2026-02-11T23:31:17.687+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.687+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.688+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bd84d997:::obj292:head, version: 107'292, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bd84d997:::obj292:head@107'292, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.688+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bc55eba4:::obj534:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bc55eba4:::obj534:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 115/115 lb 4:bc55eba4:::obj534:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:bc55eba4:::obj534:head) v3 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bbf67374:::obj187:head local-lis/les=112/113 n=452 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:bc55eba4:::obj534:head) 2026-02-11T23:31:17.688+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bc55eba4:::obj534:head) v3 finish latency 0.000012 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.688+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.688+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55de900 con 0x3e4a000 2026-02-11T23:31:17.688+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:bd84d997:::obj292:head)]) -- 0x57f0600 con 0x3e4a000 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.688+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 955 ==== MOSDPGPush(4.0 115/112 [PushOp(4:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183d40 con 0x3e4a000 2026-02-11T23:31:17.688+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.688+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.688+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 956 ==== pg_backfill(progress 4.0 e 115/115 lb 4:bd84d997:::obj292:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5600 con 0x3e4a000 2026-02-11T23:31:17.688+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:bd84d997:::obj292:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.688+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bd84d997:::obj292:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.688+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.688+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bd84d997:::obj292:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:bdc5b82e:::obj52:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:bdc5b82e:::obj52:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:bdc5b82e:::obj52:head 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:bdc5b82e:::obj52:head v 107'52 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.688+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bdc5b82e:::obj52:head, version: 107'52, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdc5b82e:::obj52:head@107'52, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.688+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.688+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bd84d997:::obj292:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.688+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bd84d997:::obj292:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:bd84d997:::obj292:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.688+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.688+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:bd84d997:::obj292:head) v3 2026-02-11T23:31:17.688+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bc55eba4:::obj534:head local-lis/les=112/113 n=453 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:bd84d997:::obj292:head) 2026-02-11T23:31:17.688+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.688+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bd84d997:::obj292:head) v3 finish latency 0.000015 2026-02-11T23:31:17.688+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.688+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.688+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.688+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55de780 con 0x3e4a000 2026-02-11T23:31:17.688+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:bdc5b82e:::obj52:head)]) -- 0x57f1080 con 0x3e4a000 2026-02-11T23:31:17.688+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.689+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 957 ==== MOSDPGPush(4.0 115/112 [PushOp(4:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3e4a000 2026-02-11T23:31:17.689+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.689+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.689+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 958 ==== pg_backfill(progress 4.0 e 115/115 lb 4:bdc5b82e:::obj52:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.689+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:bdc5b82e:::obj52:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 115 2026-02-11T23:31:17.689+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bdc5b82e:::obj52:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bdc5b82e:::obj52:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:bdf2c185:::obj359:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:bdf2c185:::obj359:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:bdf2c185:::obj359:head 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:bdf2c185:::obj359:head v 107'359 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.689+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bdf2c185:::obj359:head, version: 107'359, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bdf2c185:::obj359:head@107'359, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bdc5b82e:::obj52:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bdc5b82e:::obj52:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:bdc5b82e:::obj52:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:bdc5b82e:::obj52:head) v3 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bd84d997:::obj292:head local-lis/les=112/113 n=454 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:bdc5b82e:::obj52:head) 2026-02-11T23:31:17.689+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bdc5b82e:::obj52:head) v3 finish latency 0.000012 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.689+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55de300 con 0x3e4a000 2026-02-11T23:31:17.689+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:bdf2c185:::obj359:head)]) -- 0x4bf0f00 con 0x3e4a000 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.689+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 959 ==== MOSDPGPush(4.0 115/112 [PushOp(4:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.689+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.689+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.689+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 960 ==== pg_backfill(progress 4.0 e 115/115 lb 4:bdf2c185:::obj359:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3e4a000 2026-02-11T23:31:17.689+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:bdf2c185:::obj359:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.689+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bdf2c185:::obj359:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bdf2c185:::obj359:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:be10f7c9:::obj475:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:be10f7c9:::obj475:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:be10f7c9:::obj475:head 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:be10f7c9:::obj475:head v 107'475 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.689+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:be10f7c9:::obj475:head, version: 107'475, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be10f7c9:::obj475:head@107'475, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bdf2c185:::obj359:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bdf2c185:::obj359:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:bdf2c185:::obj359:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:bdf2c185:::obj359:head) v3 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdc5b82e:::obj52:head local-lis/les=112/113 n=455 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:bdf2c185:::obj359:head) 2026-02-11T23:31:17.689+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.689+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bdf2c185:::obj359:head) v3 finish latency 0.000016 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.689+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.689+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.689+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55de180 con 0x3e4a000 2026-02-11T23:31:17.689+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:be10f7c9:::obj475:head)]) -- 0x55de600 con 0x3e4a000 2026-02-11T23:31:17.689+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.690+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 961 ==== MOSDPGPush(4.0 115/112 [PushOp(4:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.690+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.690+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.690+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 962 ==== pg_backfill(progress 4.0 e 115/115 lb 4:be10f7c9:::obj475:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5080 con 0x3e4a000 2026-02-11T23:31:17.690+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:be10f7c9:::obj475:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.690+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:be10f7c9:::obj475:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:be10f7c9:::obj475:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:be1f0c3b:::obj423:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:be1f0c3b:::obj423:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:be1f0c3b:::obj423:head 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:be1f0c3b:::obj423:head v 107'423 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.690+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:be1f0c3b:::obj423:head, version: 107'423, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be1f0c3b:::obj423:head@107'423, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:be10f7c9:::obj475:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.690+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:be10f7c9:::obj475:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:be10f7c9:::obj475:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.690+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.690+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:be10f7c9:::obj475:head) v3 2026-02-11T23:31:17.690+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bdf2c185:::obj359:head local-lis/les=112/113 n=456 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:be10f7c9:::obj475:head) 2026-02-11T23:31:17.690+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.690+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:be10f7c9:::obj475:head) v3 finish latency 0.000012 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.690+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.690+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57f1380 con 0x3e4a000 2026-02-11T23:31:17.690+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:be1f0c3b:::obj423:head)]) -- 0x57f0300 con 0x3e4a000 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.690+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 963 ==== MOSDPGPush(4.0 115/112 [PushOp(4:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3a00 con 0x3e4a000 2026-02-11T23:31:17.690+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.690+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.690+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 964 ==== pg_backfill(progress 4.0 e 115/115 lb 4:be1f0c3b:::obj423:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3e4a000 2026-02-11T23:31:17.690+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:be1f0c3b:::obj423:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.690+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:be1f0c3b:::obj423:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:be1f0c3b:::obj423:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:be604f9b:::obj512:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:be604f9b:::obj512:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:be604f9b:::obj512:head 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:be604f9b:::obj512:head v 107'512 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.690+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.690+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:be604f9b:::obj512:head, version: 107'512, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:be604f9b:::obj512:head@107'512, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.690+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:be1f0c3b:::obj423:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.690+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:be1f0c3b:::obj423:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:be1f0c3b:::obj423:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.690+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.690+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:be1f0c3b:::obj423:head) v3 2026-02-11T23:31:17.690+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be10f7c9:::obj475:head local-lis/les=112/113 n=457 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:be1f0c3b:::obj423:head) 2026-02-11T23:31:17.690+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.690+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:be1f0c3b:::obj423:head) v3 finish latency 0.000010 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.690+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.691+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.691+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.691+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55de000 con 0x3e4a000 2026-02-11T23:31:17.691+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:be604f9b:::obj512:head)]) -- 0x4ba2d80 con 0x3e4a000 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.691+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 965 ==== MOSDPGPush(4.0 115/112 [PushOp(4:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.691+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.691+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.691+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.691+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 966 ==== pg_backfill(progress 4.0 e 115/115 lb 4:be604f9b:::obj512:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3e4a000 2026-02-11T23:31:17.691+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:be604f9b:::obj512:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.691+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:be604f9b:::obj512:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.691+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.691+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.691+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.691+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.691+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:be604f9b:::obj512:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.691+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.691+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.691+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:beac82f5:::obj51:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.691+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:beac82f5:::obj51:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.691+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:beac82f5:::obj51:head 2026-02-11T23:31:17.691+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:beac82f5:::obj51:head v 107'51 2026-02-11T23:31:17.691+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.691+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.691+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:beac82f5:::obj51:head, version: 107'51, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:beac82f5:::obj51:head@107'51, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.691+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:be604f9b:::obj512:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.691+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:be604f9b:::obj512:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:be604f9b:::obj512:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.691+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.691+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:be604f9b:::obj512:head) v3 2026-02-11T23:31:17.691+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be1f0c3b:::obj423:head local-lis/les=112/113 n=458 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:be604f9b:::obj512:head) 2026-02-11T23:31:17.691+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.691+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:be604f9b:::obj512:head) v3 finish latency 0.000010 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.691+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.691+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.691+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.691+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:beac82f5:::obj51:head)]) -- 0x57f0180 con 0x3e4a000 2026-02-11T23:31:17.691+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.692+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 967 ==== MOSDPGPush(4.0 115/112 [PushOp(4:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3e4a000 2026-02-11T23:31:17.692+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.692+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.692+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 968 ==== pg_backfill(progress 4.0 e 115/115 lb 4:beac82f5:::obj51:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3e4a000 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.692+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:beac82f5:::obj51:head) v3 prio 15 type 95 cost 0 latency 0.000007 epoch 115 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.692+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:beac82f5:::obj51:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:beac82f5:::obj51:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:bec6553a:::obj65:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:bec6553a:::obj65:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:bec6553a:::obj65:head 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:bec6553a:::obj65:head v 107'65 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.692+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bec6553a:::obj65:head, version: 107'65, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bec6553a:::obj65:head@107'65, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:beac82f5:::obj51:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:beac82f5:::obj51:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:beac82f5:::obj51:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:beac82f5:::obj51:head) v3 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:be604f9b:::obj512:head local-lis/les=112/113 n=459 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:beac82f5:::obj51:head) 2026-02-11T23:31:17.692+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:beac82f5:::obj51:head) v3 finish latency 0.000010 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.692+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57de780 con 0x3e4a000 2026-02-11T23:31:17.692+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:bec6553a:::obj65:head)]) -- 0x53b2780 con 0x3e4a000 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.692+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 969 ==== MOSDPGPush(4.0 115/112 [PushOp(4:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3e4a000 2026-02-11T23:31:17.692+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.692+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.692+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 970 ==== pg_backfill(progress 4.0 e 115/115 lb 4:bec6553a:::obj65:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3e4a000 2026-02-11T23:31:17.692+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:bec6553a:::obj65:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.692+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bec6553a:::obj65:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bec6553a:::obj65:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:bf41e297:::obj573:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:bf41e297:::obj573:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:bf41e297:::obj573:head 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:bf41e297:::obj573:head v 107'573 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.692+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bf41e297:::obj573:head, version: 107'573, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf41e297:::obj573:head@107'573, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bec6553a:::obj65:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bec6553a:::obj65:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:bec6553a:::obj65:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:bec6553a:::obj65:head) v3 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:beac82f5:::obj51:head local-lis/les=112/113 n=460 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:bec6553a:::obj65:head) 2026-02-11T23:31:17.692+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.692+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bec6553a:::obj65:head) v3 finish latency 0.000015 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.692+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.692+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.692+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57f1e00 con 0x3e4a000 2026-02-11T23:31:17.692+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:bf41e297:::obj573:head)]) -- 0x53b3b00 con 0x3e4a000 2026-02-11T23:31:17.692+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.693+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 971 ==== MOSDPGPush(4.0 115/112 [PushOp(4:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51821a0 con 0x3e4a000 2026-02-11T23:31:17.693+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.693+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.693+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 972 ==== pg_backfill(progress 4.0 e 115/115 lb 4:bf41e297:::obj573:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3e4a000 2026-02-11T23:31:17.693+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:bf41e297:::obj573:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.693+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bf41e297:::obj573:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bf41e297:::obj573:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:bf770068:::obj268:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:bf770068:::obj268:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:bf770068:::obj268:head 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:bf770068:::obj268:head v 107'268 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.693+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:bf770068:::obj268:head, version: 107'268, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:bf770068:::obj268:head@107'268, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bf41e297:::obj573:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bf41e297:::obj573:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:bf41e297:::obj573:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:bf41e297:::obj573:head) v3 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bec6553a:::obj65:head local-lis/les=112/113 n=461 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:bf41e297:::obj573:head) 2026-02-11T23:31:17.693+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bf41e297:::obj573:head) v3 finish latency 0.000015 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.693+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3c80 con 0x3e4a000 2026-02-11T23:31:17.693+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:bf770068:::obj268:head)]) -- 0x4bf1080 con 0x3e4a000 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.693+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 973 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3e4a000 2026-02-11T23:31:17.693+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.693+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.693+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 974 ==== pg_backfill(progress 4.0 e 115/115 lb 4:bf770068:::obj268:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3e4a000 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.693+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:bf770068:::obj268:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.693+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bf770068:::obj268:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bf770068:::obj268:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c0848ef4:::obj545:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c0848ef4:::obj545:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c0848ef4:::obj545:head 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c0848ef4:::obj545:head v 107'545 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.693+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.693+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c0848ef4:::obj545:head, version: 107'545, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0848ef4:::obj545:head@107'545, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:bf770068:::obj268:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bf770068:::obj268:head) v3 prio 15 cost 0 latency 0.000055 pg_backfill(progress 4.0 e 115/115 lb 4:bf770068:::obj268:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:bf770068:::obj268:head) v3 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf41e297:::obj573:head local-lis/les=112/113 n=462 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:bf770068:::obj268:head) 2026-02-11T23:31:17.693+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:bf770068:::obj268:head) v3 finish latency 0.000011 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.693+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.693+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.693+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57f1b00 con 0x3e4a000 2026-02-11T23:31:17.693+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c0848ef4:::obj545:head)]) -- 0x57dfe00 con 0x3e4a000 2026-02-11T23:31:17.693+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.694+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 975 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51831e0 con 0x3e4a000 2026-02-11T23:31:17.694+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 115 2026-02-11T23:31:17.694+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.694+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.694+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 976 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c0848ef4:::obj545:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3e4a000 2026-02-11T23:31:17.694+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c0848ef4:::obj545:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.694+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.694+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c0848ef4:::obj545:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.694+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.694+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.694+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 115/112 [PushOp(4:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.694+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.694+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.694+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c0848ef4:::obj545:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.694+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.694+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.694+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c0c49def:::obj22:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.694+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c0c49def:::obj22:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.694+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c0c49def:::obj22:head 2026-02-11T23:31:17.694+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c0c49def:::obj22:head v 107'22 2026-02-11T23:31:17.694+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.694+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.694+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c0c49def:::obj22:head, version: 107'22, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c0c49def:::obj22:head@107'22, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.694+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.694+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.694+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c0848ef4:::obj545:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.694+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c0848ef4:::obj545:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:c0848ef4:::obj545:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.694+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.694+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c0848ef4:::obj545:head) v3 2026-02-11T23:31:17.694+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:bf770068:::obj268:head local-lis/les=112/113 n=463 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c0848ef4:::obj545:head) 2026-02-11T23:31:17.694+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.694+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c0848ef4:::obj545:head) v3 finish latency 0.000012 2026-02-11T23:31:17.694+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.694+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.694+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.694+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.694+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.694+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.694+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.694+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.694+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c0c49def:::obj22:head)]) -- 0x57f0180 con 0x3e4a000 2026-02-11T23:31:17.694+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.695+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 977 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3e4a000 2026-02-11T23:31:17.695+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.695+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.695+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 978 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c0c49def:::obj22:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3e4a000 2026-02-11T23:31:17.695+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c0c49def:::obj22:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.695+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c0c49def:::obj22:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 115/112 [PushOp(4:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c0c49def:::obj22:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c138e0af:::obj576:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c138e0af:::obj576:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c138e0af:::obj576:head 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c138e0af:::obj576:head v 107'576 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.695+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c138e0af:::obj576:head, version: 107'576, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c138e0af:::obj576:head@107'576, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c0c49def:::obj22:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.695+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c0c49def:::obj22:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 115/115 lb 4:c0c49def:::obj22:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.695+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.695+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c0c49def:::obj22:head) v3 2026-02-11T23:31:17.695+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0848ef4:::obj545:head local-lis/les=112/113 n=464 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c0c49def:::obj22:head) 2026-02-11T23:31:17.695+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.695+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c0c49def:::obj22:head) v3 finish latency 0.000015 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.695+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.695+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c138e0af:::obj576:head)]) -- 0x57f1800 con 0x3e4a000 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.695+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 979 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.695+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.695+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.695+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 980 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c138e0af:::obj576:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3e4a000 2026-02-11T23:31:17.695+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c138e0af:::obj576:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.695+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c138e0af:::obj576:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c138e0af:::obj576:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c19e5add:::obj152:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c19e5add:::obj152:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c19e5add:::obj152:head 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c19e5add:::obj152:head v 107'152 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.695+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c19e5add:::obj152:head, version: 107'152, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c19e5add:::obj152:head@107'152, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c138e0af:::obj576:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.695+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c138e0af:::obj576:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:c138e0af:::obj576:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.695+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.695+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c138e0af:::obj576:head) v3 2026-02-11T23:31:17.695+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c0c49def:::obj22:head local-lis/les=112/113 n=465 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c138e0af:::obj576:head) 2026-02-11T23:31:17.695+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.695+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c138e0af:::obj576:head) v3 finish latency 0.000014 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.695+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.695+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.695+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57df080 con 0x3e4a000 2026-02-11T23:31:17.695+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c19e5add:::obj152:head)]) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.695+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.696+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 981 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43f3a00 con 0x3e4a000 2026-02-11T23:31:17.696+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.696+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.696+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.696+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 982 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c19e5add:::obj152:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3e4a000 2026-02-11T23:31:17.696+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c19e5add:::obj152:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.696+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.696+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c19e5add:::obj152:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.696+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.696+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.696+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.696+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.696+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.696+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c19e5add:::obj152:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.696+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.696+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.696+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c275c16a:::obj32:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.696+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c275c16a:::obj32:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.696+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c275c16a:::obj32:head 2026-02-11T23:31:17.696+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c275c16a:::obj32:head v 107'32 2026-02-11T23:31:17.696+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.696+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.696+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c275c16a:::obj32:head, version: 107'32, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c275c16a:::obj32:head@107'32, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.696+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.696+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.696+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c19e5add:::obj152:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.696+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c19e5add:::obj152:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:c19e5add:::obj152:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.696+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.696+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c19e5add:::obj152:head) v3 2026-02-11T23:31:17.696+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c138e0af:::obj576:head local-lis/les=112/113 n=466 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c19e5add:::obj152:head) 2026-02-11T23:31:17.696+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.696+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c19e5add:::obj152:head) v3 finish latency 0.000013 2026-02-11T23:31:17.696+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.696+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.696+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.696+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.696+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.696+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.696+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.696+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4f50180 con 0x3e4a000 2026-02-11T23:31:17.696+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c275c16a:::obj32:head)]) -- 0x4d8f800 con 0x3e4a000 2026-02-11T23:31:17.696+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.697+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 983 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.697+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.697+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.697+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 984 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c275c16a:::obj32:head) ==== 1061+0+0 (unknown 0 0 0) 0x4ded080 con 0x3e4a000 2026-02-11T23:31:17.697+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c275c16a:::obj32:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 115 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.697+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c275c16a:::obj32:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c275c16a:::obj32:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c290fff4:::obj320:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c290fff4:::obj320:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c290fff4:::obj320:head 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c290fff4:::obj320:head v 107'320 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.697+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c290fff4:::obj320:head, version: 107'320, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c290fff4:::obj320:head@107'320, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c275c16a:::obj32:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c275c16a:::obj32:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:c275c16a:::obj32:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c275c16a:::obj32:head) v3 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c19e5add:::obj152:head local-lis/les=112/113 n=467 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c275c16a:::obj32:head) 2026-02-11T23:31:17.697+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c275c16a:::obj32:head) v3 finish latency 0.000010 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.697+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6000 con 0x3e4a000 2026-02-11T23:31:17.697+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c290fff4:::obj320:head)]) -- 0x4ba2d80 con 0x3e4a000 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.697+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 985 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.697+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.697+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.697+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 986 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c290fff4:::obj320:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3e4a000 2026-02-11T23:31:17.697+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c290fff4:::obj320:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.697+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c290fff4:::obj320:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c290fff4:::obj320:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c3033278:::obj208:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c3033278:::obj208:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c3033278:::obj208:head 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c3033278:::obj208:head v 107'208 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.697+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3033278:::obj208:head, version: 107'208, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3033278:::obj208:head@107'208, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c290fff4:::obj320:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c290fff4:::obj320:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:c290fff4:::obj320:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c290fff4:::obj320:head) v3 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c275c16a:::obj32:head local-lis/les=112/113 n=468 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c290fff4:::obj320:head) 2026-02-11T23:31:17.697+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.697+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c290fff4:::obj320:head) v3 finish latency 0.000017 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.697+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.697+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.697+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6480 con 0x3e4a000 2026-02-11T23:31:17.697+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c3033278:::obj208:head)]) -- 0x55f6180 con 0x3e4a000 2026-02-11T23:31:17.697+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.698+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 987 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3e4a000 2026-02-11T23:31:17.698+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.698+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.698+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 988 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c3033278:::obj208:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3e4a000 2026-02-11T23:31:17.698+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3033278:::obj208:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.698+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3033278:::obj208:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.698+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.698+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3033278:::obj208:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c3ae426e:::obj537:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c3ae426e:::obj537:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c3ae426e:::obj537:head 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c3ae426e:::obj537:head v 107'537 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.698+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3ae426e:::obj537:head, version: 107'537, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ae426e:::obj537:head@107'537, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.698+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.698+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3033278:::obj208:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.698+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3033278:::obj208:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:c3033278:::obj208:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.698+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.698+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c3033278:::obj208:head) v3 2026-02-11T23:31:17.698+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c290fff4:::obj320:head local-lis/les=112/113 n=469 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c3033278:::obj208:head) 2026-02-11T23:31:17.698+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.698+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3033278:::obj208:head) v3 finish latency 0.000015 2026-02-11T23:31:17.698+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.698+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.698+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6900 con 0x3e4a000 2026-02-11T23:31:17.698+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c3ae426e:::obj537:head)]) -- 0x55f6600 con 0x3e4a000 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.698+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 989 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x5183d40 con 0x3e4a000 2026-02-11T23:31:17.698+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.698+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.698+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 990 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c3ae426e:::obj537:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3e4a000 2026-02-11T23:31:17.698+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3ae426e:::obj537:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.698+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3ae426e:::obj537:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.698+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.698+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3ae426e:::obj537:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c3ba172d:::obj29:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c3ba172d:::obj29:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c3ba172d:::obj29:head 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c3ba172d:::obj29:head v 107'29 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.698+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.698+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3ba172d:::obj29:head, version: 107'29, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3ba172d:::obj29:head@107'29, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.698+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.698+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.698+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3ae426e:::obj537:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.698+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3ae426e:::obj537:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:c3ae426e:::obj537:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.698+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.698+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c3ae426e:::obj537:head) v3 2026-02-11T23:31:17.698+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3033278:::obj208:head local-lis/les=112/113 n=470 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c3ae426e:::obj537:head) 2026-02-11T23:31:17.698+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.698+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3ae426e:::obj537:head) v3 finish latency 0.000016 2026-02-11T23:31:17.698+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.699+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.699+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.699+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6c00 con 0x3e4a000 2026-02-11T23:31:17.699+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c3ba172d:::obj29:head)]) -- 0x55f6a80 con 0x3e4a000 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.699+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 991 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3520 con 0x3e4a000 2026-02-11T23:31:17.699+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.699+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.699+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.699+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 992 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c3ba172d:::obj29:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3e4a000 2026-02-11T23:31:17.699+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3ba172d:::obj29:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.699+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.699+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3ba172d:::obj29:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.699+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.699+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.699+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3ba172d:::obj29:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.699+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.699+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.699+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.699+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c3bae056:::obj148:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.699+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c3bae056:::obj148:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.699+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c3bae056:::obj148:head 2026-02-11T23:31:17.699+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c3bae056:::obj148:head v 107'148 2026-02-11T23:31:17.699+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.699+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.699+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3bae056:::obj148:head, version: 107'148, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bae056:::obj148:head@107'148, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.699+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3ba172d:::obj29:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.699+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3ba172d:::obj29:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 115/115 lb 4:c3ba172d:::obj29:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.699+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.699+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c3ba172d:::obj29:head) v3 2026-02-11T23:31:17.699+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ae426e:::obj537:head local-lis/les=112/113 n=471 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c3ba172d:::obj29:head) 2026-02-11T23:31:17.699+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.699+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3ba172d:::obj29:head) v3 finish latency 0.000012 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.699+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.699+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.699+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6d80 con 0x3e4a000 2026-02-11T23:31:17.699+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c3bae056:::obj148:head)]) -- 0x57f0300 con 0x3e4a000 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.699+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 993 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dfe1a0 con 0x3e4a000 2026-02-11T23:31:17.699+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.699+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.699+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 994 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c3bae056:::obj148:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd180 con 0x3e4a000 2026-02-11T23:31:17.699+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3bae056:::obj148:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.699+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3bae056:::obj148:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.699+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.699+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.699+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.699+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.699+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3bae056:::obj148:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.699+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.699+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c3bcd33c:::obj192:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c3bcd33c:::obj192:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c3bcd33c:::obj192:head 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c3bcd33c:::obj192:head v 107'192 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.700+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c3bcd33c:::obj192:head, version: 107'192, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c3bcd33c:::obj192:head@107'192, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.700+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.700+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3bae056:::obj148:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3bae056:::obj148:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 115/115 lb 4:c3bae056:::obj148:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c3bae056:::obj148:head) v3 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3ba172d:::obj29:head local-lis/les=112/113 n=472 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c3bae056:::obj148:head) 2026-02-11T23:31:17.700+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3bae056:::obj148:head) v3 finish latency 0.000016 2026-02-11T23:31:17.700+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.700+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.700+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7200 con 0x3e4a000 2026-02-11T23:31:17.700+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c3bcd33c:::obj192:head)]) -- 0x55f6f00 con 0x3e4a000 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.700+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 995 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2b60 con 0x3e4a000 2026-02-11T23:31:17.700+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.700+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.700+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.700+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 996 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c3bcd33c:::obj192:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def180 con 0x3e4a000 2026-02-11T23:31:17.700+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3bcd33c:::obj192:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.700+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.700+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3bcd33c:::obj192:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.700+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.700+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3bcd33c:::obj192:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c41d6f6c:::obj298:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.700+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c41d6f6c:::obj298:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c41d6f6c:::obj298:head 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c41d6f6c:::obj298:head v 107'298 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.700+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.700+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c41d6f6c:::obj298:head, version: 107'298, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c41d6f6c:::obj298:head@107'298, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.700+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c3bcd33c:::obj192:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3bcd33c:::obj192:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:c3bcd33c:::obj192:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c3bcd33c:::obj192:head) v3 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bae056:::obj148:head local-lis/les=112/113 n=473 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c3bcd33c:::obj192:head) 2026-02-11T23:31:17.700+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c3bcd33c:::obj192:head) v3 finish latency 0.000012 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.700+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.700+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.700+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7380 con 0x3e4a000 2026-02-11T23:31:17.700+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c41d6f6c:::obj298:head)]) -- 0x55de600 con 0x3e4a000 2026-02-11T23:31:17.700+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.701+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 997 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2680 con 0x3e4a000 2026-02-11T23:31:17.701+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.701+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.701+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 998 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c41d6f6c:::obj298:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3e4a000 2026-02-11T23:31:17.701+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c41d6f6c:::obj298:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.701+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c41d6f6c:::obj298:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.701+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.701+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c41d6f6c:::obj298:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c4f18846:::obj81:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c4f18846:::obj81:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c4f18846:::obj81:head 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c4f18846:::obj81:head v 107'81 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.701+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c4f18846:::obj81:head, version: 107'81, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c4f18846:::obj81:head@107'81, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.701+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.701+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c41d6f6c:::obj298:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.701+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c41d6f6c:::obj298:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:c41d6f6c:::obj298:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.701+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.701+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c41d6f6c:::obj298:head) v3 2026-02-11T23:31:17.701+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c3bcd33c:::obj192:head local-lis/les=112/113 n=474 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c41d6f6c:::obj298:head) 2026-02-11T23:31:17.701+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.701+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c41d6f6c:::obj298:head) v3 finish latency 0.000015 2026-02-11T23:31:17.701+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.701+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.701+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7680 con 0x3e4a000 2026-02-11T23:31:17.701+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c4f18846:::obj81:head)]) -- 0x55f7500 con 0x3e4a000 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.701+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 999 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x51a21a0 con 0x3e4a000 2026-02-11T23:31:17.701+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.701+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.701+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1000 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c4f18846:::obj81:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164c00 con 0x3e4a000 2026-02-11T23:31:17.701+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c4f18846:::obj81:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.701+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c4f18846:::obj81:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.701+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.701+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c4f18846:::obj81:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c52c9666:::obj5:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c52c9666:::obj5:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c52c9666:::obj5:head 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c52c9666:::obj5:head v 107'5 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.701+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.701+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c52c9666:::obj5:head, version: 107'5, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c52c9666:::obj5:head@107'5, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.701+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.701+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.701+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c4f18846:::obj81:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.701+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c4f18846:::obj81:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:c4f18846:::obj81:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.701+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.701+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c4f18846:::obj81:head) v3 2026-02-11T23:31:17.701+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c41d6f6c:::obj298:head local-lis/les=112/113 n=475 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c4f18846:::obj81:head) 2026-02-11T23:31:17.701+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.701+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c4f18846:::obj81:head) v3 finish latency 0.000015 2026-02-11T23:31:17.701+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.702+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7980 con 0x3e4a000 2026-02-11T23:31:17.702+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c52c9666:::obj5:head)]) -- 0x55f7800 con 0x3e4a000 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.702+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1001 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.702+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.702+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.702+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1002 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c52c9666:::obj5:head) ==== 1060+0+0 (unknown 0 0 0) 0x5165180 con 0x3e4a000 2026-02-11T23:31:17.702+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c52c9666:::obj5:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.702+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c52c9666:::obj5:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c52c9666:::obj5:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c57e020a:::obj395:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c57e020a:::obj395:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c57e020a:::obj395:head 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c57e020a:::obj395:head v 107'395 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.702+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c57e020a:::obj395:head, version: 107'395, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c57e020a:::obj395:head@107'395, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c52c9666:::obj5:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.702+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c52c9666:::obj5:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:c52c9666:::obj5:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.702+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.702+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c52c9666:::obj5:head) v3 2026-02-11T23:31:17.702+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c4f18846:::obj81:head local-lis/les=112/113 n=476 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c52c9666:::obj5:head) 2026-02-11T23:31:17.702+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.702+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c52c9666:::obj5:head) v3 finish latency 0.000016 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.702+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7e00 con 0x3e4a000 2026-02-11T23:31:17.702+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c57e020a:::obj395:head)]) -- 0x55f7b00 con 0x3e4a000 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.702+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1003 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.702+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.702+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.702+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1004 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c57e020a:::obj395:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3e4a000 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.702+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c57e020a:::obj395:head) v3 prio 15 type 95 cost 0 latency 0.000006 epoch 115 2026-02-11T23:31:17.702+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c57e020a:::obj395:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c57e020a:::obj395:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c59209e1:::obj523:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c59209e1:::obj523:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c59209e1:::obj523:head 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c59209e1:::obj523:head v 107'523 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.702+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.702+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c59209e1:::obj523:head, version: 107'523, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c59209e1:::obj523:head@107'523, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.702+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c57e020a:::obj395:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.702+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c57e020a:::obj395:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 115/115 lb 4:c57e020a:::obj395:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.702+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.702+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c57e020a:::obj395:head) v3 2026-02-11T23:31:17.702+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c52c9666:::obj5:head local-lis/les=112/113 n=477 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c57e020a:::obj395:head) 2026-02-11T23:31:17.702+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.702+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c57e020a:::obj395:head) v3 finish latency 0.000014 2026-02-11T23:31:17.702+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.703+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.703+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.703+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7680 con 0x3e4a000 2026-02-11T23:31:17.703+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c59209e1:::obj523:head)]) -- 0x55f7980 con 0x3e4a000 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.703+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1005 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.703+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.703+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.703+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1006 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c59209e1:::obj523:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.703+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c59209e1:::obj523:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.703+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c59209e1:::obj523:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.703+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.703+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c59209e1:::obj523:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.703+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.703+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.703+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.703+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.703+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c5df6474:::obj459:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c5df6474:::obj459:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.703+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c5df6474:::obj459:head 2026-02-11T23:31:17.703+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c5df6474:::obj459:head v 107'459 2026-02-11T23:31:17.703+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.703+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.703+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c5df6474:::obj459:head, version: 107'459, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5df6474:::obj459:head@107'459, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.703+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.703+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c59209e1:::obj523:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.703+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c59209e1:::obj523:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:c59209e1:::obj523:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.703+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.703+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c59209e1:::obj523:head) v3 2026-02-11T23:31:17.703+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c57e020a:::obj395:head local-lis/les=112/113 n=478 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c59209e1:::obj523:head) 2026-02-11T23:31:17.703+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.703+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c59209e1:::obj523:head) v3 finish latency 0.000015 2026-02-11T23:31:17.703+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.703+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.703+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.703+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6f00 con 0x3e4a000 2026-02-11T23:31:17.703+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c5df6474:::obj459:head)]) -- 0x55de600 con 0x3e4a000 2026-02-11T23:31:17.703+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.704+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1007 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.704+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.704+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.704+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1008 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c5df6474:::obj459:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.704+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c5df6474:::obj459:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.704+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c5df6474:::obj459:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c5df6474:::obj459:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c5e569a1:::obj444:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c5e569a1:::obj444:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c5e569a1:::obj444:head 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c5e569a1:::obj444:head v 107'444 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.704+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c5e569a1:::obj444:head, version: 107'444, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c5e569a1:::obj444:head@107'444, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c5df6474:::obj459:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c5df6474:::obj459:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:c5df6474:::obj459:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c5df6474:::obj459:head) v3 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c59209e1:::obj523:head local-lis/les=112/113 n=479 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c5df6474:::obj459:head) 2026-02-11T23:31:17.704+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c5df6474:::obj459:head) v3 finish latency 0.000014 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.704+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6d80 con 0x3e4a000 2026-02-11T23:31:17.704+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c5e569a1:::obj444:head)]) -- 0x55f7200 con 0x3e4a000 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.704+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1009 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183040 con 0x3e4a000 2026-02-11T23:31:17.704+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.704+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.704+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1010 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c5e569a1:::obj444:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.704+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c5e569a1:::obj444:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.704+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c5e569a1:::obj444:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c5e569a1:::obj444:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c64b60bf:::obj563:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c64b60bf:::obj563:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c64b60bf:::obj563:head 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c64b60bf:::obj563:head v 107'563 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.704+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.704+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c64b60bf:::obj563:head, version: 107'563, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c64b60bf:::obj563:head@107'563, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c5e569a1:::obj444:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c5e569a1:::obj444:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:c5e569a1:::obj444:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c5e569a1:::obj444:head) v3 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5df6474:::obj459:head local-lis/les=112/113 n=480 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c5e569a1:::obj444:head) 2026-02-11T23:31:17.704+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c5e569a1:::obj444:head) v3 finish latency 0.000012 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.704+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.704+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.704+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6a80 con 0x3e4a000 2026-02-11T23:31:17.704+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c64b60bf:::obj563:head)]) -- 0x4bf0f00 con 0x3e4a000 2026-02-11T23:31:17.704+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.705+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1011 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4680 con 0x3e4a000 2026-02-11T23:31:17.705+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.705+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.705+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1012 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c64b60bf:::obj563:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.705+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c64b60bf:::obj563:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.705+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c64b60bf:::obj563:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.705+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.705+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c64b60bf:::obj563:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c68dea80:::obj235:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c68dea80:::obj235:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c68dea80:::obj235:head 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c68dea80:::obj235:head v 107'235 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.705+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c68dea80:::obj235:head, version: 107'235, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c68dea80:::obj235:head@107'235, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.705+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.705+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c64b60bf:::obj563:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.705+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c64b60bf:::obj563:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:c64b60bf:::obj563:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.705+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.705+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c64b60bf:::obj563:head) v3 2026-02-11T23:31:17.705+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c5e569a1:::obj444:head local-lis/les=112/113 n=481 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c64b60bf:::obj563:head) 2026-02-11T23:31:17.705+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.705+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c64b60bf:::obj563:head) v3 finish latency 0.000016 2026-02-11T23:31:17.705+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.705+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.705+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6900 con 0x3e4a000 2026-02-11T23:31:17.705+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c68dea80:::obj235:head)]) -- 0x55f6c00 con 0x3e4a000 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.705+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1013 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516da00 con 0x3e4a000 2026-02-11T23:31:17.705+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.705+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.705+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1014 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c68dea80:::obj235:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.705+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c68dea80:::obj235:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.705+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c68dea80:::obj235:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.705+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.705+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c68dea80:::obj235:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c71bdb5a:::obj349:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c71bdb5a:::obj349:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c71bdb5a:::obj349:head 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c71bdb5a:::obj349:head v 107'349 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.705+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.705+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c71bdb5a:::obj349:head, version: 107'349, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c71bdb5a:::obj349:head@107'349, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.705+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.705+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.705+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c68dea80:::obj235:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.705+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c68dea80:::obj235:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:c68dea80:::obj235:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.705+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.705+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c68dea80:::obj235:head) v3 2026-02-11T23:31:17.705+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c64b60bf:::obj563:head local-lis/les=112/113 n=482 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c68dea80:::obj235:head) 2026-02-11T23:31:17.705+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.705+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c68dea80:::obj235:head) v3 finish latency 0.000015 2026-02-11T23:31:17.705+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.706+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.706+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.706+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ba2d80 con 0x3e4a000 2026-02-11T23:31:17.706+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c71bdb5a:::obj349:head)]) -- 0x55f6180 con 0x3e4a000 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.706+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1015 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182000 con 0x3e4a000 2026-02-11T23:31:17.706+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.706+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.706+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1016 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c71bdb5a:::obj349:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.706+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c71bdb5a:::obj349:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.706+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c71bdb5a:::obj349:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.706+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.706+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.706+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.706+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c71bdb5a:::obj349:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.706+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.706+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.706+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c7481937:::obj131:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c7481937:::obj131:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.706+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c7481937:::obj131:head 2026-02-11T23:31:17.706+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c7481937:::obj131:head v 107'131 2026-02-11T23:31:17.706+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.706+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.706+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c7481937:::obj131:head, version: 107'131, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c7481937:::obj131:head@107'131, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.706+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.706+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c71bdb5a:::obj349:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.706+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c71bdb5a:::obj349:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:c71bdb5a:::obj349:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.706+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.706+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c71bdb5a:::obj349:head) v3 2026-02-11T23:31:17.706+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c68dea80:::obj235:head local-lis/les=112/113 n=483 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c71bdb5a:::obj349:head) 2026-02-11T23:31:17.706+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.706+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c71bdb5a:::obj349:head) v3 finish latency 0.000015 2026-02-11T23:31:17.706+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.706+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.706+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.706+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4f50180 con 0x3e4a000 2026-02-11T23:31:17.706+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c7481937:::obj131:head)]) -- 0x55f6000 con 0x3e4a000 2026-02-11T23:31:17.706+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.706+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1017 ==== MOSDPGPush(4.0 115/112 [PushOp(4:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51824e0 con 0x3e4a000 2026-02-11T23:31:17.706+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.706+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.707+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1018 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c7481937:::obj131:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.707+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c7481937:::obj131:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.707+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c7481937:::obj131:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 115/112 [PushOp(4:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c7481937:::obj131:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:c8495ae6:::obj149:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:c8495ae6:::obj149:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:c8495ae6:::obj149:head 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:c8495ae6:::obj149:head v 107'149 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.707+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:c8495ae6:::obj149:head, version: 107'149, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:c8495ae6:::obj149:head@107'149, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c7481937:::obj131:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c7481937:::obj131:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 115/115 lb 4:c7481937:::obj131:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c7481937:::obj131:head) v3 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c71bdb5a:::obj349:head local-lis/les=112/113 n=484 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c7481937:::obj131:head) 2026-02-11T23:31:17.707+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c7481937:::obj131:head) v3 finish latency 0.000012 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.707+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.707+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:c8495ae6:::obj149:head)]) -- 0x57f1080 con 0x3e4a000 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.707+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1019 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3e4a000 2026-02-11T23:31:17.707+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.707+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.707+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1020 ==== pg_backfill(progress 4.0 e 115/115 lb 4:c8495ae6:::obj149:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163b80 con 0x3e4a000 2026-02-11T23:31:17.707+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:c8495ae6:::obj149:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.707+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c8495ae6:::obj149:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c8495ae6:::obj149:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ca82ed30:::obj135:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ca82ed30:::obj135:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ca82ed30:::obj135:head 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ca82ed30:::obj135:head v 107'135 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.707+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ca82ed30:::obj135:head, version: 107'135, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ca82ed30:::obj135:head@107'135, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:c8495ae6:::obj149:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c8495ae6:::obj149:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:c8495ae6:::obj149:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:c8495ae6:::obj149:head) v3 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c7481937:::obj131:head local-lis/les=112/113 n=485 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:c8495ae6:::obj149:head) 2026-02-11T23:31:17.707+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.707+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:c8495ae6:::obj149:head) v3 finish latency 0.000019 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.707+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.707+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.707+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.707+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ca82ed30:::obj135:head)]) -- 0x57df080 con 0x3e4a000 2026-02-11T23:31:17.707+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.708+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1021 ==== MOSDPGPush(4.0 115/112 [PushOp(4:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2680 con 0x3e4a000 2026-02-11T23:31:17.708+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.708+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.708+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1022 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ca82ed30:::obj135:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163600 con 0x3e4a000 2026-02-11T23:31:17.708+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ca82ed30:::obj135:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.708+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ca82ed30:::obj135:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ca82ed30:::obj135:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:cadaa6e3:::obj220:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:cadaa6e3:::obj220:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:cadaa6e3:::obj220:head 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:cadaa6e3:::obj220:head v 107'220 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.708+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cadaa6e3:::obj220:head, version: 107'220, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cadaa6e3:::obj220:head@107'220, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ca82ed30:::obj135:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ca82ed30:::obj135:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 115/115 lb 4:ca82ed30:::obj135:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ca82ed30:::obj135:head) v3 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:c8495ae6:::obj149:head local-lis/les=112/113 n=486 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ca82ed30:::obj135:head) 2026-02-11T23:31:17.708+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ca82ed30:::obj135:head) v3 finish latency 0.000011 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.708+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57f0180 con 0x3e4a000 2026-02-11T23:31:17.708+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:cadaa6e3:::obj220:head)]) -- 0x57f0600 con 0x3e4a000 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.708+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1023 ==== MOSDPGPush(4.0 115/112 [PushOp(4:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.708+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.708+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.708+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1024 ==== pg_backfill(progress 4.0 e 115/115 lb 4:cadaa6e3:::obj220:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163080 con 0x3e4a000 2026-02-11T23:31:17.708+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:cadaa6e3:::obj220:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.708+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cadaa6e3:::obj220:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cadaa6e3:::obj220:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:cb1a5635:::obj233:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:cb1a5635:::obj233:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:cb1a5635:::obj233:head 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:cb1a5635:::obj233:head v 107'233 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.708+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb1a5635:::obj233:head, version: 107'233, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1a5635:::obj233:head@107'233, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cadaa6e3:::obj220:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cadaa6e3:::obj220:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:cadaa6e3:::obj220:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:cadaa6e3:::obj220:head) v3 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ca82ed30:::obj135:head local-lis/les=112/113 n=487 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:cadaa6e3:::obj220:head) 2026-02-11T23:31:17.708+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.708+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cadaa6e3:::obj220:head) v3 finish latency 0.000015 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.708+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.708+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.708+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57f1b00 con 0x3e4a000 2026-02-11T23:31:17.708+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:cb1a5635:::obj233:head)]) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.708+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.709+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1025 ==== MOSDPGPush(4.0 115/112 [PushOp(4:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.709+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.709+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.709+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1026 ==== pg_backfill(progress 4.0 e 115/115 lb 4:cb1a5635:::obj233:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3e4a000 2026-02-11T23:31:17.709+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb1a5635:::obj233:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.709+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb1a5635:::obj233:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.709+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.709+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb1a5635:::obj233:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:cb1be4ca:::obj406:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:cb1be4ca:::obj406:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:cb1be4ca:::obj406:head 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:cb1be4ca:::obj406:head v 107'406 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.709+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb1be4ca:::obj406:head, version: 107'406, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb1be4ca:::obj406:head@107'406, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.709+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.709+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb1a5635:::obj233:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.709+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb1a5635:::obj233:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:cb1a5635:::obj233:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.709+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.709+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:cb1a5635:::obj233:head) v3 2026-02-11T23:31:17.709+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cadaa6e3:::obj220:head local-lis/les=112/113 n=488 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:cb1a5635:::obj233:head) 2026-02-11T23:31:17.709+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.709+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb1a5635:::obj233:head) v3 finish latency 0.000014 2026-02-11T23:31:17.709+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.709+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.709+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x53b3b00 con 0x3e4a000 2026-02-11T23:31:17.709+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:cb1be4ca:::obj406:head)]) -- 0x4bf1080 con 0x3e4a000 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.709+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1027 ==== MOSDPGPush(4.0 115/112 [PushOp(4:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.709+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.709+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.709+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1028 ==== pg_backfill(progress 4.0 e 115/115 lb 4:cb1be4ca:::obj406:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162580 con 0x3e4a000 2026-02-11T23:31:17.709+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb1be4ca:::obj406:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.709+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb1be4ca:::obj406:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.709+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.709+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb1be4ca:::obj406:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:cb4c4ca4:::obj480:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:cb4c4ca4:::obj480:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:cb4c4ca4:::obj480:head 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:cb4c4ca4:::obj480:head v 107'480 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.709+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.709+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb4c4ca4:::obj480:head, version: 107'480, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb4c4ca4:::obj480:head@107'480, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.709+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.709+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.709+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb1be4ca:::obj406:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.709+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb1be4ca:::obj406:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:cb1be4ca:::obj406:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.709+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.709+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:cb1be4ca:::obj406:head) v3 2026-02-11T23:31:17.709+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1a5635:::obj233:head local-lis/les=112/113 n=489 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:cb1be4ca:::obj406:head) 2026-02-11T23:31:17.709+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.709+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb1be4ca:::obj406:head) v3 finish latency 0.000015 2026-02-11T23:31:17.709+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.710+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.710+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.710+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5610000 con 0x3e4a000 2026-02-11T23:31:17.710+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:cb4c4ca4:::obj480:head)]) -- 0x57f1e00 con 0x3e4a000 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.710+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1029 ==== MOSDPGPush(4.0 115/112 [PushOp(4:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.710+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.710+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.710+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1030 ==== pg_backfill(progress 4.0 e 115/115 lb 4:cb4c4ca4:::obj480:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162000 con 0x3e4a000 2026-02-11T23:31:17.710+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb4c4ca4:::obj480:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.710+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb4c4ca4:::obj480:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.710+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.710+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.710+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.710+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb4c4ca4:::obj480:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.710+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.710+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.710+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:cb843a8e:::obj34:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:cb843a8e:::obj34:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.710+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:cb843a8e:::obj34:head 2026-02-11T23:31:17.710+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:cb843a8e:::obj34:head v 107'34 2026-02-11T23:31:17.710+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.710+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.710+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cb843a8e:::obj34:head, version: 107'34, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cb843a8e:::obj34:head@107'34, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.710+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.710+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb4c4ca4:::obj480:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.710+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb4c4ca4:::obj480:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:cb4c4ca4:::obj480:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.710+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.710+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:cb4c4ca4:::obj480:head) v3 2026-02-11T23:31:17.710+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb1be4ca:::obj406:head local-lis/les=112/113 n=490 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:cb4c4ca4:::obj480:head) 2026-02-11T23:31:17.710+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.710+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb4c4ca4:::obj480:head) v3 finish latency 0.000015 2026-02-11T23:31:17.710+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.710+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.710+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.710+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5610300 con 0x3e4a000 2026-02-11T23:31:17.710+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:cb843a8e:::obj34:head)]) -- 0x5610180 con 0x3e4a000 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.710+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1031 ==== MOSDPGPush(4.0 115/112 [PushOp(4:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2340 con 0x3e4a000 2026-02-11T23:31:17.710+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.710+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.710+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1032 ==== pg_backfill(progress 4.0 e 115/115 lb 4:cb843a8e:::obj34:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3e4a000 2026-02-11T23:31:17.710+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb843a8e:::obj34:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.710+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb843a8e:::obj34:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.710+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 115/112 [PushOp(4:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb843a8e:::obj34:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:cbce4a0b:::obj95:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:cbce4a0b:::obj95:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:cbce4a0b:::obj95:head 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:cbce4a0b:::obj95:head v 107'95 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.711+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cbce4a0b:::obj95:head, version: 107'95, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cbce4a0b:::obj95:head@107'95, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cb843a8e:::obj34:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.711+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb843a8e:::obj34:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 115/115 lb 4:cb843a8e:::obj34:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.711+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.711+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:cb843a8e:::obj34:head) v3 2026-02-11T23:31:17.711+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb4c4ca4:::obj480:head local-lis/les=112/113 n=491 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:cb843a8e:::obj34:head) 2026-02-11T23:31:17.711+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.711+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cb843a8e:::obj34:head) v3 finish latency 0.000015 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.711+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5610600 con 0x3e4a000 2026-02-11T23:31:17.711+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:cbce4a0b:::obj95:head)]) -- 0x5610480 con 0x3e4a000 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.711+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1033 ==== MOSDPGPush(4.0 115/112 [PushOp(4:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2820 con 0x3e4a000 2026-02-11T23:31:17.711+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.711+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.711+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1034 ==== pg_backfill(progress 4.0 e 115/115 lb 4:cbce4a0b:::obj95:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3e4a000 2026-02-11T23:31:17.711+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:cbce4a0b:::obj95:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.711+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cbce4a0b:::obj95:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cbce4a0b:::obj95:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:cc461e5d:::obj45:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:cc461e5d:::obj45:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:cc461e5d:::obj45:head 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:cc461e5d:::obj45:head v 107'45 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.711+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cc461e5d:::obj45:head, version: 107'45, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cc461e5d:::obj45:head@107'45, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cbce4a0b:::obj95:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.711+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cbce4a0b:::obj95:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:cbce4a0b:::obj95:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.711+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.711+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:cbce4a0b:::obj95:head) v3 2026-02-11T23:31:17.711+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cb843a8e:::obj34:head local-lis/les=112/113 n=492 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:cbce4a0b:::obj95:head) 2026-02-11T23:31:17.711+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.711+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cbce4a0b:::obj95:head) v3 finish latency 0.000015 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.711+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.711+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.711+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5610900 con 0x3e4a000 2026-02-11T23:31:17.711+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:cc461e5d:::obj45:head)]) -- 0x5610780 con 0x3e4a000 2026-02-11T23:31:17.711+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.712+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1035 ==== MOSDPGPush(4.0 115/112 [PushOp(4:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3e4a000 2026-02-11T23:31:17.712+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.712+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.712+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1036 ==== pg_backfill(progress 4.0 e 115/115 lb 4:cc461e5d:::obj45:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3e4a000 2026-02-11T23:31:17.712+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:cc461e5d:::obj45:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.712+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cc461e5d:::obj45:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cc461e5d:::obj45:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:cdb46b38:::obj588:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:cdb46b38:::obj588:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:cdb46b38:::obj588:head 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:cdb46b38:::obj588:head v 107'588 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.712+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cdb46b38:::obj588:head, version: 107'588, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cdb46b38:::obj588:head@107'588, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cc461e5d:::obj45:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.712+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cc461e5d:::obj45:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:cc461e5d:::obj45:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.712+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.712+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:cc461e5d:::obj45:head) v3 2026-02-11T23:31:17.712+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cbce4a0b:::obj95:head local-lis/les=112/113 n=493 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:cc461e5d:::obj45:head) 2026-02-11T23:31:17.712+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.712+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cc461e5d:::obj45:head) v3 finish latency 0.000014 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.712+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5610d80 con 0x3e4a000 2026-02-11T23:31:17.712+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:cdb46b38:::obj588:head)]) -- 0x5610a80 con 0x3e4a000 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.712+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1037 ==== MOSDPGPush(4.0 115/112 [PushOp(4:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3e4a000 2026-02-11T23:31:17.712+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.712+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.712+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1038 ==== pg_backfill(progress 4.0 e 115/115 lb 4:cdb46b38:::obj588:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3e4a000 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.712+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:cdb46b38:::obj588:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.712+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cdb46b38:::obj588:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cdb46b38:::obj588:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:cde49336:::obj263:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:cde49336:::obj263:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:cde49336:::obj263:head 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:cde49336:::obj263:head v 107'263 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.712+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cde49336:::obj263:head, version: 107'263, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cde49336:::obj263:head@107'263, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cdb46b38:::obj588:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.712+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cdb46b38:::obj588:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 115/115 lb 4:cdb46b38:::obj588:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.712+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.712+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:cdb46b38:::obj588:head) v3 2026-02-11T23:31:17.712+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cc461e5d:::obj45:head local-lis/les=112/113 n=494 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:cdb46b38:::obj588:head) 2026-02-11T23:31:17.712+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.712+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cdb46b38:::obj588:head) v3 finish latency 0.000015 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.712+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.712+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.712+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.712+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5611200 con 0x3e4a000 2026-02-11T23:31:17.713+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:cde49336:::obj263:head)]) -- 0x5610f00 con 0x3e4a000 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.713+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1039 ==== MOSDPGPush(4.0 115/112 [PushOp(4:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3e4a000 2026-02-11T23:31:17.713+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.713+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.713+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1040 ==== pg_backfill(progress 4.0 e 115/115 lb 4:cde49336:::obj263:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5bb80 con 0x3e4a000 2026-02-11T23:31:17.713+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:cde49336:::obj263:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.713+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cde49336:::obj263:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.713+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 115/112 [PushOp(4:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.713+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cde49336:::obj263:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.713+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.713+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.713+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:cf1e3413:::obj224:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:cf1e3413:::obj224:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.713+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:cf1e3413:::obj224:head 2026-02-11T23:31:17.713+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:cf1e3413:::obj224:head v 107'224 2026-02-11T23:31:17.713+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.713+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.713+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:cf1e3413:::obj224:head, version: 107'224, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:cf1e3413:::obj224:head@107'224, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000071 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cde49336:::obj263:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cde49336:::obj263:head) v3 prio 15 cost 0 latency 0.000089 pg_backfill(progress 4.0 e 115/115 lb 4:cde49336:::obj263:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:cde49336:::obj263:head) v3 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cdb46b38:::obj588:head local-lis/les=112/113 n=495 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:cde49336:::obj263:head) 2026-02-11T23:31:17.713+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cde49336:::obj263:head) v3 finish latency 0.000011 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.713+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.713+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5611680 con 0x3e4a000 2026-02-11T23:31:17.713+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:cf1e3413:::obj224:head)]) -- 0x5611380 con 0x3e4a000 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.713+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1041 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ebba0 con 0x3e4a000 2026-02-11T23:31:17.713+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.713+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.713+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1042 ==== pg_backfill(progress 4.0 e 115/115 lb 4:cf1e3413:::obj224:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3e4a000 2026-02-11T23:31:17.713+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:cf1e3413:::obj224:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.713+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cf1e3413:::obj224:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.713+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cf1e3413:::obj224:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d05ea685:::obj543:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.713+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d05ea685:::obj543:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d05ea685:::obj543:head 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d05ea685:::obj543:head v 107'543 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.713+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.713+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d05ea685:::obj543:head, version: 107'543, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d05ea685:::obj543:head@107'543, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.714+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:cf1e3413:::obj224:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cf1e3413:::obj224:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:cf1e3413:::obj224:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:cf1e3413:::obj224:head) v3 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cde49336:::obj263:head local-lis/les=112/113 n=496 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:cf1e3413:::obj224:head) 2026-02-11T23:31:17.714+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:cf1e3413:::obj224:head) v3 finish latency 0.000016 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.714+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.714+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5611800 con 0x3e4a000 2026-02-11T23:31:17.714+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d05ea685:::obj543:head)]) -- 0x55dea80 con 0x3e4a000 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.714+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1043 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.714+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.714+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.714+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1044 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d05ea685:::obj543:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3e4a000 2026-02-11T23:31:17.714+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d05ea685:::obj543:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.714+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d05ea685:::obj543:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 115/112 [PushOp(4:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.714+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.714+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d05ea685:::obj543:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d06c69cb:::obj532:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d06c69cb:::obj532:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d06c69cb:::obj532:head 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d06c69cb:::obj532:head v 107'532 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.714+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d06c69cb:::obj532:head, version: 107'532, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d06c69cb:::obj532:head@107'532, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.714+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.714+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d05ea685:::obj543:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.714+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d05ea685:::obj543:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 115/115 lb 4:d05ea685:::obj543:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.714+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.714+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d05ea685:::obj543:head) v3 2026-02-11T23:31:17.714+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:cf1e3413:::obj224:head local-lis/les=112/113 n=497 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d05ea685:::obj543:head) 2026-02-11T23:31:17.714+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.714+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d05ea685:::obj543:head) v3 finish latency 0.000015 2026-02-11T23:31:17.714+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.714+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.714+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.714+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5611c80 con 0x3e4a000 2026-02-11T23:31:17.714+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d06c69cb:::obj532:head)]) -- 0x5611980 con 0x3e4a000 2026-02-11T23:31:17.714+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.715+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1045 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.715+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.715+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.715+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1046 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d06c69cb:::obj532:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3e4a000 2026-02-11T23:31:17.715+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d06c69cb:::obj532:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.715+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d06c69cb:::obj532:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d06c69cb:::obj532:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d0e799f1:::obj299:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d0e799f1:::obj299:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d0e799f1:::obj299:head 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d0e799f1:::obj299:head v 107'299 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.715+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d0e799f1:::obj299:head, version: 107'299, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0e799f1:::obj299:head@107'299, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d06c69cb:::obj532:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.715+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d06c69cb:::obj532:head) v3 prio 15 cost 0 latency 0.000077 pg_backfill(progress 4.0 e 115/115 lb 4:d06c69cb:::obj532:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.715+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.715+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d06c69cb:::obj532:head) v3 2026-02-11T23:31:17.715+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d05ea685:::obj543:head local-lis/les=112/113 n=498 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d06c69cb:::obj532:head) 2026-02-11T23:31:17.715+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.715+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d06c69cb:::obj532:head) v3 finish latency 0.000012 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.715+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5610180 con 0x3e4a000 2026-02-11T23:31:17.715+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d0e799f1:::obj299:head)]) -- 0x5611e00 con 0x3e4a000 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.715+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1047 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.715+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.715+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.715+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1048 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d0e799f1:::obj299:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3e4a000 2026-02-11T23:31:17.715+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d0e799f1:::obj299:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.715+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d0e799f1:::obj299:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d0e799f1:::obj299:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d0f049f1:::obj387:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d0f049f1:::obj387:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d0f049f1:::obj387:head 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d0f049f1:::obj387:head v 107'387 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.715+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d0f049f1:::obj387:head, version: 107'387, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d0f049f1:::obj387:head@107'387, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000043 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d0e799f1:::obj299:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.715+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d0e799f1:::obj299:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 115/115 lb 4:d0e799f1:::obj299:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.715+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.715+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d0e799f1:::obj299:head) v3 2026-02-11T23:31:17.715+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d06c69cb:::obj532:head local-lis/les=112/113 n=499 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d0e799f1:::obj299:head) 2026-02-11T23:31:17.715+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.715+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d0e799f1:::obj299:head) v3 finish latency 0.000009 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.715+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.715+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.715+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5610000 con 0x3e4a000 2026-02-11T23:31:17.715+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d0f049f1:::obj387:head)]) -- 0x5610300 con 0x3e4a000 2026-02-11T23:31:17.715+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.716+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1049 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.716+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.716+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.716+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1050 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d0f049f1:::obj387:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3e4a000 2026-02-11T23:31:17.716+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d0f049f1:::obj387:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.716+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d0f049f1:::obj387:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d0f049f1:::obj387:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d11b9b34:::obj309:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d11b9b34:::obj309:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d11b9b34:::obj309:head 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d11b9b34:::obj309:head v 107'309 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.716+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d11b9b34:::obj309:head, version: 107'309, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d11b9b34:::obj309:head@107'309, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d0f049f1:::obj387:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d0f049f1:::obj387:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:d0f049f1:::obj387:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d0f049f1:::obj387:head) v3 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0e799f1:::obj299:head local-lis/les=112/113 n=500 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d0f049f1:::obj387:head) 2026-02-11T23:31:17.716+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d0f049f1:::obj387:head) v3 finish latency 0.000009 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.716+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.716+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d11b9b34:::obj309:head)]) -- 0x4bf1080 con 0x3e4a000 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.716+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1051 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43f2680 con 0x3e4a000 2026-02-11T23:31:17.716+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 115 2026-02-11T23:31:17.716+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.716+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1052 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d11b9b34:::obj309:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3e4a000 2026-02-11T23:31:17.716+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d11b9b34:::obj309:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.716+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d11b9b34:::obj309:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d11b9b34:::obj309:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d1337354:::obj10:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d1337354:::obj10:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d1337354:::obj10:head 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d1337354:::obj10:head v 107'10 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.716+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.716+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d1337354:::obj10:head, version: 107'10, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1337354:::obj10:head@107'10, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.716+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d11b9b34:::obj309:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d11b9b34:::obj309:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 115/115 lb 4:d11b9b34:::obj309:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d11b9b34:::obj309:head) v3 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d0f049f1:::obj387:head local-lis/les=112/113 n=501 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d11b9b34:::obj309:head) 2026-02-11T23:31:17.716+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.716+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d11b9b34:::obj309:head) v3 finish latency 0.000013 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.716+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.717+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.717+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.717+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57f1b00 con 0x3e4a000 2026-02-11T23:31:17.717+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d1337354:::obj10:head)]) -- 0x57f0780 con 0x3e4a000 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.717+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1053 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4dcba00 con 0x3e4a000 2026-02-11T23:31:17.717+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.717+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.717+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.717+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1054 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d1337354:::obj10:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3e4a000 2026-02-11T23:31:17.717+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.717+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d1337354:::obj10:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.717+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d1337354:::obj10:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.717+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.717+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.717+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 115/112 [PushOp(4:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.717+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d1337354:::obj10:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.717+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.717+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.717+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d1c8135f:::obj67:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.717+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d1c8135f:::obj67:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.717+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d1c8135f:::obj67:head 2026-02-11T23:31:17.717+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d1c8135f:::obj67:head v 107'67 2026-02-11T23:31:17.717+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.717+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.717+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d1c8135f:::obj67:head, version: 107'67, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d1c8135f:::obj67:head@107'67, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.717+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d1337354:::obj10:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.717+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d1337354:::obj10:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:d1337354:::obj10:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.717+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.717+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d1337354:::obj10:head) v3 2026-02-11T23:31:17.717+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d11b9b34:::obj309:head local-lis/les=112/113 n=502 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d1337354:::obj10:head) 2026-02-11T23:31:17.717+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.717+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d1337354:::obj10:head) v3 finish latency 0.000013 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.717+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.717+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.717+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57f0600 con 0x3e4a000 2026-02-11T23:31:17.717+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d1c8135f:::obj67:head)]) -- 0x55def00 con 0x3e4a000 2026-02-11T23:31:17.717+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.718+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1055 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51824e0 con 0x3e4a000 2026-02-11T23:31:17.718+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.718+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.718+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1056 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d1c8135f:::obj67:head) ==== 1061+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3e4a000 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.718+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d1c8135f:::obj67:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.718+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d1c8135f:::obj67:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 115/112 [PushOp(4:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d1c8135f:::obj67:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d33f403b:::obj434:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d33f403b:::obj434:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d33f403b:::obj434:head 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d33f403b:::obj434:head v 107'434 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.718+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d33f403b:::obj434:head, version: 107'434, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d33f403b:::obj434:head@107'434, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d1c8135f:::obj67:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d1c8135f:::obj67:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 115/115 lb 4:d1c8135f:::obj67:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d1c8135f:::obj67:head) v3 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1337354:::obj10:head local-lis/les=112/113 n=503 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d1c8135f:::obj67:head) 2026-02-11T23:31:17.718+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d1c8135f:::obj67:head) v3 finish latency 0.000011 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.718+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57f0180 con 0x3e4a000 2026-02-11T23:31:17.718+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d33f403b:::obj434:head)]) -- 0x4e5d200 con 0x3e4a000 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.718+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1057 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182000 con 0x3e4a000 2026-02-11T23:31:17.718+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.718+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.718+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1058 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d33f403b:::obj434:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3e4a000 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.718+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d33f403b:::obj434:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.718+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d33f403b:::obj434:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d33f403b:::obj434:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d3f6948e:::obj218:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d3f6948e:::obj218:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d3f6948e:::obj218:head 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d3f6948e:::obj218:head v 107'218 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.718+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.718+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d3f6948e:::obj218:head, version: 107'218, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d3f6948e:::obj218:head@107'218, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d33f403b:::obj434:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d33f403b:::obj434:head) v3 prio 15 cost 0 latency 0.000060 pg_backfill(progress 4.0 e 115/115 lb 4:d33f403b:::obj434:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d33f403b:::obj434:head) v3 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d1c8135f:::obj67:head local-lis/les=112/113 n=504 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d33f403b:::obj434:head) 2026-02-11T23:31:17.718+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d33f403b:::obj434:head) v3 finish latency 0.000010 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.718+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.718+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.718+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57df080 con 0x3e4a000 2026-02-11T23:31:17.718+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d3f6948e:::obj218:head)]) -- 0x4bf1200 con 0x3e4a000 2026-02-11T23:31:17.718+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.719+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1059 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516da00 con 0x3e4a000 2026-02-11T23:31:17.719+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.719+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.719+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1060 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d3f6948e:::obj218:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3e4a000 2026-02-11T23:31:17.719+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d3f6948e:::obj218:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.719+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d3f6948e:::obj218:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d3f6948e:::obj218:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 115/112 [PushOp(4:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d4006106:::obj329:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d4006106:::obj329:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d4006106:::obj329:head 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d4006106:::obj329:head v 107'329 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.719+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d4006106:::obj329:head, version: 107'329, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4006106:::obj329:head@107'329, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d3f6948e:::obj218:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.719+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d3f6948e:::obj218:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:d3f6948e:::obj218:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.719+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.719+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d3f6948e:::obj218:head) v3 2026-02-11T23:31:17.719+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d33f403b:::obj434:head local-lis/les=112/113 n=505 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d3f6948e:::obj218:head) 2026-02-11T23:31:17.719+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.719+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d3f6948e:::obj218:head) v3 finish latency 0.000011 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.719+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.719+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.719+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d4006106:::obj329:head)]) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.719+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1061 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4680 con 0x3e4a000 2026-02-11T23:31:17.719+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.719+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.719+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1062 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d4006106:::obj329:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bc100 con 0x3e4a000 2026-02-11T23:31:17.719+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d4006106:::obj329:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.719+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d4006106:::obj329:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d4006106:::obj329:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d43593c2:::obj365:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d43593c2:::obj365:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d43593c2:::obj365:head 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d43593c2:::obj365:head v 107'365 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.719+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.719+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d43593c2:::obj365:head, version: 107'365, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d43593c2:::obj365:head@107'365, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.719+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d4006106:::obj329:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.719+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d4006106:::obj329:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 115/115 lb 4:d4006106:::obj329:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.719+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.719+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d4006106:::obj329:head) v3 2026-02-11T23:31:17.719+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d3f6948e:::obj218:head local-lis/les=112/113 n=506 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d4006106:::obj329:head) 2026-02-11T23:31:17.719+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.719+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d4006106:::obj329:head) v3 finish latency 0.000012 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.719+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.720+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.720+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.720+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57f1080 con 0x3e4a000 2026-02-11T23:31:17.720+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d43593c2:::obj365:head)]) -- 0x55dfc80 con 0x3e4a000 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.720+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1063 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183040 con 0x3e4a000 2026-02-11T23:31:17.720+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.720+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.720+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1064 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d43593c2:::obj365:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3e4a000 2026-02-11T23:31:17.720+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d43593c2:::obj365:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.720+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d43593c2:::obj365:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.720+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.720+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d43593c2:::obj365:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.720+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.720+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.720+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.720+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.720+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d4c3bba7:::obj106:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d4c3bba7:::obj106:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.720+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d4c3bba7:::obj106:head 2026-02-11T23:31:17.720+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d4c3bba7:::obj106:head v 107'106 2026-02-11T23:31:17.720+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.720+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.720+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d4c3bba7:::obj106:head, version: 107'106, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d4c3bba7:::obj106:head@107'106, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000061 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.720+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.720+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d43593c2:::obj365:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.720+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d43593c2:::obj365:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 115/115 lb 4:d43593c2:::obj365:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.720+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.720+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d43593c2:::obj365:head) v3 2026-02-11T23:31:17.720+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4006106:::obj329:head local-lis/les=112/113 n=507 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d43593c2:::obj365:head) 2026-02-11T23:31:17.720+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.720+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d43593c2:::obj365:head) v3 finish latency 0.000012 2026-02-11T23:31:17.720+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.720+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.720+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.720+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4f50180 con 0x3e4a000 2026-02-11T23:31:17.720+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d4c3bba7:::obj106:head)]) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.720+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.721+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1065 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.721+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.721+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.721+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1066 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d4c3bba7:::obj106:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.721+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d4c3bba7:::obj106:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.721+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d4c3bba7:::obj106:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 115/112 [PushOp(4:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d4c3bba7:::obj106:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d5f69272:::obj407:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d5f69272:::obj407:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d5f69272:::obj407:head 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d5f69272:::obj407:head v 107'407 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.721+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d5f69272:::obj407:head, version: 107'407, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d5f69272:::obj407:head@107'407, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d4c3bba7:::obj106:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d4c3bba7:::obj106:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:d4c3bba7:::obj106:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d4c3bba7:::obj106:head) v3 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d43593c2:::obj365:head local-lis/les=112/113 n=508 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d4c3bba7:::obj106:head) 2026-02-11T23:31:17.721+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d4c3bba7:::obj106:head) v3 finish latency 0.000014 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.721+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6180 con 0x3e4a000 2026-02-11T23:31:17.721+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d5f69272:::obj407:head)]) -- 0x4bf0a80 con 0x3e4a000 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.721+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1067 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.721+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.721+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.721+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1068 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d5f69272:::obj407:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3e4a000 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.721+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d5f69272:::obj407:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.721+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d5f69272:::obj407:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d5f69272:::obj407:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d6df9426:::obj469:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d6df9426:::obj469:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d6df9426:::obj469:head 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d6df9426:::obj469:head v 107'469 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.721+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.721+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d6df9426:::obj469:head, version: 107'469, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d6df9426:::obj469:head@107'469, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000045 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d5f69272:::obj407:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d5f69272:::obj407:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 115/115 lb 4:d5f69272:::obj407:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d5f69272:::obj407:head) v3 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d4c3bba7:::obj106:head local-lis/les=112/113 n=509 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d5f69272:::obj407:head) 2026-02-11T23:31:17.721+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d5f69272:::obj407:head) v3 finish latency 0.000012 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.721+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.721+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.721+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ba2d80 con 0x3e4a000 2026-02-11T23:31:17.721+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d6df9426:::obj469:head)]) -- 0x55df680 con 0x3e4a000 2026-02-11T23:31:17.721+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.722+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1069 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.722+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.722+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.722+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.722+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1070 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d6df9426:::obj469:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.722+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.722+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d6df9426:::obj469:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.722+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d6df9426:::obj469:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.722+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.722+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.722+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.722+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.722+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.722+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d6df9426:::obj469:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.722+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.722+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.722+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d80bb902:::obj418:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.722+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d80bb902:::obj418:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.722+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d80bb902:::obj418:head 2026-02-11T23:31:17.722+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d80bb902:::obj418:head v 107'418 2026-02-11T23:31:17.722+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.722+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.722+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d80bb902:::obj418:head, version: 107'418, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d80bb902:::obj418:head@107'418, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.722+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.722+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.722+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d6df9426:::obj469:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.722+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d6df9426:::obj469:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:d6df9426:::obj469:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.722+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.722+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d6df9426:::obj469:head) v3 2026-02-11T23:31:17.722+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d5f69272:::obj407:head local-lis/les=112/113 n=510 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d6df9426:::obj469:head) 2026-02-11T23:31:17.722+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.722+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d6df9426:::obj469:head) v3 finish latency 0.000015 2026-02-11T23:31:17.722+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.722+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.722+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.722+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.722+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.722+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.722+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.722+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6c00 con 0x3e4a000 2026-02-11T23:31:17.722+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d80bb902:::obj418:head)]) -- 0x55df200 con 0x3e4a000 2026-02-11T23:31:17.722+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.723+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1071 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.723+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.723+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.723+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1072 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d80bb902:::obj418:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.723+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d80bb902:::obj418:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.723+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d80bb902:::obj418:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 115/112 [PushOp(4:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d80bb902:::obj418:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d83b1cbf:::obj272:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d83b1cbf:::obj272:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d83b1cbf:::obj272:head 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d83b1cbf:::obj272:head v 107'272 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.723+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d83b1cbf:::obj272:head, version: 107'272, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d83b1cbf:::obj272:head@107'272, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d80bb902:::obj418:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d80bb902:::obj418:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:d80bb902:::obj418:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d80bb902:::obj418:head) v3 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d6df9426:::obj469:head local-lis/les=112/113 n=511 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d80bb902:::obj418:head) 2026-02-11T23:31:17.723+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d80bb902:::obj418:head) v3 finish latency 0.000018 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.723+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6900 con 0x3e4a000 2026-02-11T23:31:17.723+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d83b1cbf:::obj272:head)]) -- 0x55dfc80 con 0x3e4a000 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.723+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1073 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3e4a000 2026-02-11T23:31:17.723+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 115 2026-02-11T23:31:17.723+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.723+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1074 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d83b1cbf:::obj272:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.723+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d83b1cbf:::obj272:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.723+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d83b1cbf:::obj272:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 115/112 [PushOp(4:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d83b1cbf:::obj272:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d85a09a6:::obj441:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d85a09a6:::obj441:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d85a09a6:::obj441:head 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d85a09a6:::obj441:head v 107'441 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.723+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.723+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d85a09a6:::obj441:head, version: 107'441, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d85a09a6:::obj441:head@107'441, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000046 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d83b1cbf:::obj272:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.723+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d83b1cbf:::obj272:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:d83b1cbf:::obj272:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d83b1cbf:::obj272:head) v3 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d80bb902:::obj418:head local-lis/les=112/113 n=512 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d83b1cbf:::obj272:head) 2026-02-11T23:31:17.723+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.723+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d83b1cbf:::obj272:head) v3 finish latency 0.000022 2026-02-11T23:31:17.723+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.724+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.724+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.724+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7200 con 0x3e4a000 2026-02-11T23:31:17.724+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d85a09a6:::obj441:head)]) -- 0x4bf0f00 con 0x3e4a000 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.724+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1075 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3e4a000 2026-02-11T23:31:17.724+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.724+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.724+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1076 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d85a09a6:::obj441:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.724+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d85a09a6:::obj441:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.724+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d85a09a6:::obj441:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.724+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.724+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.724+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d85a09a6:::obj441:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.724+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.724+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.724+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.724+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d881fc7d:::obj242:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d881fc7d:::obj242:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.724+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d881fc7d:::obj242:head 2026-02-11T23:31:17.724+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d881fc7d:::obj242:head v 107'242 2026-02-11T23:31:17.724+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.724+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.724+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d881fc7d:::obj242:head, version: 107'242, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d881fc7d:::obj242:head@107'242, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.724+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.724+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d85a09a6:::obj441:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.724+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d85a09a6:::obj441:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:d85a09a6:::obj441:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.724+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.724+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d85a09a6:::obj441:head) v3 2026-02-11T23:31:17.724+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d83b1cbf:::obj272:head local-lis/les=112/113 n=513 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d85a09a6:::obj441:head) 2026-02-11T23:31:17.724+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.724+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d85a09a6:::obj441:head) v3 finish latency 0.000020 2026-02-11T23:31:17.724+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.724+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.724+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.724+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6f00 con 0x3e4a000 2026-02-11T23:31:17.724+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d881fc7d:::obj242:head)]) -- 0x55f6d80 con 0x3e4a000 2026-02-11T23:31:17.724+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.725+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1077 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.725+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.725+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.725+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1078 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d881fc7d:::obj242:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.725+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d881fc7d:::obj242:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.725+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d881fc7d:::obj242:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d881fc7d:::obj242:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d9239af3:::obj409:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d9239af3:::obj409:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d9239af3:::obj409:head 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d9239af3:::obj409:head v 107'409 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.725+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d9239af3:::obj409:head, version: 107'409, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9239af3:::obj409:head@107'409, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d881fc7d:::obj242:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.725+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d881fc7d:::obj242:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:d881fc7d:::obj242:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.725+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.725+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d881fc7d:::obj242:head) v3 2026-02-11T23:31:17.725+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d85a09a6:::obj441:head local-lis/les=112/113 n=514 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d881fc7d:::obj242:head) 2026-02-11T23:31:17.725+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.725+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d881fc7d:::obj242:head) v3 finish latency 0.000025 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.725+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7b00 con 0x3e4a000 2026-02-11T23:31:17.725+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d9239af3:::obj409:head)]) -- 0x55f7980 con 0x3e4a000 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.725+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1079 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3e4a000 2026-02-11T23:31:17.725+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 115 2026-02-11T23:31:17.725+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.725+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1080 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d9239af3:::obj409:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.725+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d9239af3:::obj409:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.725+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d9239af3:::obj409:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000035 MOSDPGPush(4.0 115/112 [PushOp(4:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d9239af3:::obj409:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d99b7c45:::obj564:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d99b7c45:::obj564:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d99b7c45:::obj564:head 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d99b7c45:::obj564:head v 107'564 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.725+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.725+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d99b7c45:::obj564:head, version: 107'564, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d99b7c45:::obj564:head@107'564, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.725+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d9239af3:::obj409:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.725+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d9239af3:::obj409:head) v3 prio 15 cost 0 latency 0.000083 pg_backfill(progress 4.0 e 115/115 lb 4:d9239af3:::obj409:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.725+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.725+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d9239af3:::obj409:head) v3 2026-02-11T23:31:17.725+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d881fc7d:::obj242:head local-lis/les=112/113 n=515 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d9239af3:::obj409:head) 2026-02-11T23:31:17.725+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.725+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d9239af3:::obj409:head) v3 finish latency 0.000023 2026-02-11T23:31:17.725+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.726+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.726+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.726+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562a000 con 0x3e4a000 2026-02-11T23:31:17.726+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d99b7c45:::obj564:head)]) -- 0x55f7e00 con 0x3e4a000 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.726+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1081 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.726+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.726+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.726+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1082 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d99b7c45:::obj564:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.726+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d99b7c45:::obj564:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.726+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d99b7c45:::obj564:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.726+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 115/112 [PushOp(4:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.726+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.726+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.726+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.726+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d99b7c45:::obj564:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.726+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.726+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d9c98c7a:::obj262:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d9c98c7a:::obj262:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.726+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d9c98c7a:::obj262:head 2026-02-11T23:31:17.726+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d9c98c7a:::obj262:head v 107'262 2026-02-11T23:31:17.726+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.726+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.726+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d9c98c7a:::obj262:head, version: 107'262, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9c98c7a:::obj262:head@107'262, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.726+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.726+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d99b7c45:::obj564:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.726+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d99b7c45:::obj564:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 115/115 lb 4:d99b7c45:::obj564:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.726+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.726+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d99b7c45:::obj564:head) v3 2026-02-11T23:31:17.726+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9239af3:::obj409:head local-lis/les=112/113 n=516 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d99b7c45:::obj564:head) 2026-02-11T23:31:17.726+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.726+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d99b7c45:::obj564:head) v3 finish latency 0.000022 2026-02-11T23:31:17.726+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.726+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.726+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.726+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562a480 con 0x3e4a000 2026-02-11T23:31:17.726+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d9c98c7a:::obj262:head)]) -- 0x562a180 con 0x3e4a000 2026-02-11T23:31:17.726+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.727+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1083 ==== MOSDPGPush(4.0 115/112 [PushOp(4:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3e4a000 2026-02-11T23:31:17.727+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000011 epoch 115 2026-02-11T23:31:17.727+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.727+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1084 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d9c98c7a:::obj262:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165700 con 0x3e4a000 2026-02-11T23:31:17.727+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d9c98c7a:::obj262:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.727+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d9c98c7a:::obj262:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 115/112 [PushOp(4:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d9c98c7a:::obj262:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:d9da6b63:::obj302:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:d9da6b63:::obj302:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:d9da6b63:::obj302:head 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:d9da6b63:::obj302:head v 107'302 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.727+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:d9da6b63:::obj302:head, version: 107'302, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:d9da6b63:::obj302:head@107'302, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d9c98c7a:::obj262:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d9c98c7a:::obj262:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 115/115 lb 4:d9c98c7a:::obj262:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d9c98c7a:::obj262:head) v3 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d99b7c45:::obj564:head local-lis/les=112/113 n=517 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d9c98c7a:::obj262:head) 2026-02-11T23:31:17.727+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d9c98c7a:::obj262:head) v3 finish latency 0.000017 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.727+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562a900 con 0x3e4a000 2026-02-11T23:31:17.727+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:d9da6b63:::obj302:head)]) -- 0x562a600 con 0x3e4a000 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.727+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1085 ==== MOSDPGPush(4.0 115/112 [PushOp(4:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a2000 con 0x3e4a000 2026-02-11T23:31:17.727+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 115 2026-02-11T23:31:17.727+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.727+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1086 ==== pg_backfill(progress 4.0 e 115/115 lb 4:d9da6b63:::obj302:head) ==== 1062+0+0 (unknown 0 0 0) 0x5165180 con 0x3e4a000 2026-02-11T23:31:17.727+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:d9da6b63:::obj302:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.727+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d9da6b63:::obj302:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000031 MOSDPGPush(4.0 115/112 [PushOp(4:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d9da6b63:::obj302:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:da0e2ac9:::obj69:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:da0e2ac9:::obj69:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:da0e2ac9:::obj69:head 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:da0e2ac9:::obj69:head v 107'69 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.727+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.727+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:da0e2ac9:::obj69:head, version: 107'69, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da0e2ac9:::obj69:head@107'69, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:d9da6b63:::obj302:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d9da6b63:::obj302:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:d9da6b63:::obj302:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:d9da6b63:::obj302:head) v3 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9c98c7a:::obj262:head local-lis/les=112/113 n=518 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:d9da6b63:::obj302:head) 2026-02-11T23:31:17.727+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:d9da6b63:::obj302:head) v3 finish latency 0.000013 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.727+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.727+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.727+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562aa80 con 0x3e4a000 2026-02-11T23:31:17.727+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:da0e2ac9:::obj69:head)]) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.727+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.728+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1087 ==== MOSDPGPush(4.0 115/112 [PushOp(4:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.728+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000012 epoch 115 2026-02-11T23:31:17.728+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.728+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1088 ==== pg_backfill(progress 4.0 e 115/115 lb 4:da0e2ac9:::obj69:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164c00 con 0x3e4a000 2026-02-11T23:31:17.728+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:da0e2ac9:::obj69:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.728+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:da0e2ac9:::obj69:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000034 MOSDPGPush(4.0 115/112 [PushOp(4:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:da0e2ac9:::obj69:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:da920364:::obj185:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:da920364:::obj185:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:da920364:::obj185:head 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:da920364:::obj185:head v 107'185 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.728+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:da920364:::obj185:head, version: 107'185, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da920364:::obj185:head@107'185, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:da0e2ac9:::obj69:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.728+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:da0e2ac9:::obj69:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:da0e2ac9:::obj69:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.728+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.728+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:da0e2ac9:::obj69:head) v3 2026-02-11T23:31:17.728+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:d9da6b63:::obj302:head local-lis/les=112/113 n=519 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:da0e2ac9:::obj69:head) 2026-02-11T23:31:17.728+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.728+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:da0e2ac9:::obj69:head) v3 finish latency 0.000010 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.728+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.728+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562ac00 con 0x3e4a000 2026-02-11T23:31:17.728+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:da920364:::obj185:head)]) -- 0x511b980 con 0x3e4a000 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.728+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1089 ==== MOSDPGPush(4.0 115/112 [PushOp(4:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.728+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 115 2026-02-11T23:31:17.728+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.728+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1090 ==== pg_backfill(progress 4.0 e 115/115 lb 4:da920364:::obj185:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dec580 con 0x3e4a000 2026-02-11T23:31:17.728+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:da920364:::obj185:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.728+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:da920364:::obj185:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 115/112 [PushOp(4:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:da920364:::obj185:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:da932e1e:::obj333:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:da932e1e:::obj333:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:da932e1e:::obj333:head 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:da932e1e:::obj333:head v 107'333 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.728+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.728+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:da932e1e:::obj333:head, version: 107'333, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:da932e1e:::obj333:head@107'333, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000066 2026-02-11T23:31:17.728+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:da920364:::obj185:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.728+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:da920364:::obj185:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 115/115 lb 4:da920364:::obj185:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.728+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.728+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:da920364:::obj185:head) v3 2026-02-11T23:31:17.728+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da0e2ac9:::obj69:head local-lis/les=112/113 n=520 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:da920364:::obj185:head) 2026-02-11T23:31:17.728+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.728+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:da920364:::obj185:head) v3 finish latency 0.000021 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.728+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.729+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.729+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.729+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562ad80 con 0x3e4a000 2026-02-11T23:31:17.729+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:da932e1e:::obj333:head)]) -- 0x57df080 con 0x3e4a000 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.729+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1091 ==== MOSDPGPush(4.0 115/112 [PushOp(4:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3a00 con 0x3e4a000 2026-02-11T23:31:17.729+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.729+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.729+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.729+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1092 ==== pg_backfill(progress 4.0 e 115/115 lb 4:da932e1e:::obj333:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def180 con 0x3e4a000 2026-02-11T23:31:17.729+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:da932e1e:::obj333:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.729+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.729+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:da932e1e:::obj333:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.729+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.729+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.729+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.729+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:da932e1e:::obj333:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.729+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.729+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.729+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:dbca42a3:::obj382:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.729+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:dbca42a3:::obj382:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.729+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:dbca42a3:::obj382:head 2026-02-11T23:31:17.729+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:dbca42a3:::obj382:head v 107'382 2026-02-11T23:31:17.729+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.729+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.729+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dbca42a3:::obj382:head, version: 107'382, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dbca42a3:::obj382:head@107'382, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.729+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:da932e1e:::obj333:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.729+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:da932e1e:::obj333:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:da932e1e:::obj333:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.729+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.729+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:da932e1e:::obj333:head) v3 2026-02-11T23:31:17.729+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da920364:::obj185:head local-lis/les=112/113 n=521 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:da932e1e:::obj333:head) 2026-02-11T23:31:17.729+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.729+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:da932e1e:::obj333:head) v3 finish latency 0.000032 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.729+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.729+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.729+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562af00 con 0x3e4a000 2026-02-11T23:31:17.729+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:dbca42a3:::obj382:head)]) -- 0x57f0180 con 0x3e4a000 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.729+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.729+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.730+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1093 ==== MOSDPGPush(4.0 115/112 [PushOp(4:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.730+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.730+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.730+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1094 ==== pg_backfill(progress 4.0 e 115/115 lb 4:dbca42a3:::obj382:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bd180 con 0x3e4a000 2026-02-11T23:31:17.730+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:dbca42a3:::obj382:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.730+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dbca42a3:::obj382:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dbca42a3:::obj382:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:dc49c339:::obj378:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:dc49c339:::obj378:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:dc49c339:::obj378:head 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:dc49c339:::obj378:head v 107'378 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.730+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc49c339:::obj378:head, version: 107'378, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc49c339:::obj378:head@107'378, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dbca42a3:::obj382:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dbca42a3:::obj382:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:dbca42a3:::obj382:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:dbca42a3:::obj382:head) v3 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:da932e1e:::obj333:head local-lis/les=112/113 n=522 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:dbca42a3:::obj382:head) 2026-02-11T23:31:17.730+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dbca42a3:::obj382:head) v3 finish latency 0.000014 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.730+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562b080 con 0x3e4a000 2026-02-11T23:31:17.730+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:dc49c339:::obj378:head)]) -- 0x57f0600 con 0x3e4a000 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.730+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1095 ==== MOSDPGPush(4.0 115/112 [PushOp(4:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3e4a000 2026-02-11T23:31:17.730+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.730+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.730+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1096 ==== pg_backfill(progress 4.0 e 115/115 lb 4:dc49c339:::obj378:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3e4a000 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.730+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc49c339:::obj378:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.730+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc49c339:::obj378:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc49c339:::obj378:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:dc6ed3c7:::obj113:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:dc6ed3c7:::obj113:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:dc6ed3c7:::obj113:head 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:dc6ed3c7:::obj113:head v 107'113 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.730+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc6ed3c7:::obj113:head, version: 107'113, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6ed3c7:::obj113:head@107'113, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc49c339:::obj378:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc49c339:::obj378:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 115/115 lb 4:dc49c339:::obj378:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:dc49c339:::obj378:head) v3 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dbca42a3:::obj382:head local-lis/les=112/113 n=523 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:dc49c339:::obj378:head) 2026-02-11T23:31:17.730+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.730+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc49c339:::obj378:head) v3 finish latency 0.000021 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.730+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.730+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.730+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562b500 con 0x3e4a000 2026-02-11T23:31:17.730+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:dc6ed3c7:::obj113:head)]) -- 0x562b200 con 0x3e4a000 2026-02-11T23:31:17.730+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.731+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1097 ==== MOSDPGPush(4.0 115/112 [PushOp(4:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51831e0 con 0x3e4a000 2026-02-11T23:31:17.731+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.731+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.731+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.731+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1098 ==== pg_backfill(progress 4.0 e 115/115 lb 4:dc6ed3c7:::obj113:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee680 con 0x3e4a000 2026-02-11T23:31:17.731+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc6ed3c7:::obj113:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.731+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc6ed3c7:::obj113:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.731+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.731+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.731+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.731+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000033 MOSDPGPush(4.0 115/112 [PushOp(4:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.731+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc6ed3c7:::obj113:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.731+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.731+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.731+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:dc6fad89:::obj337:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.731+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:dc6fad89:::obj337:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.731+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:dc6fad89:::obj337:head 2026-02-11T23:31:17.731+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:dc6fad89:::obj337:head v 107'337 2026-02-11T23:31:17.731+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.731+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.731+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc6fad89:::obj337:head, version: 107'337, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc6fad89:::obj337:head@107'337, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000072 2026-02-11T23:31:17.731+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc6ed3c7:::obj113:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.731+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc6ed3c7:::obj113:head) v3 prio 15 cost 0 latency 0.000093 pg_backfill(progress 4.0 e 115/115 lb 4:dc6ed3c7:::obj113:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.731+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.731+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:dc6ed3c7:::obj113:head) v3 2026-02-11T23:31:17.731+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc49c339:::obj378:head local-lis/les=112/113 n=524 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:dc6ed3c7:::obj113:head) 2026-02-11T23:31:17.731+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.731+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc6ed3c7:::obj113:head) v3 finish latency 0.000017 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.731+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.731+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.731+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562b680 con 0x3e4a000 2026-02-11T23:31:17.731+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:dc6fad89:::obj337:head)]) -- 0x57f1b00 con 0x3e4a000 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.731+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1099 ==== MOSDPGPush(4.0 115/112 [PushOp(4:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3e4a000 2026-02-11T23:31:17.731+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.731+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.731+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1100 ==== pg_backfill(progress 4.0 e 115/115 lb 4:dc6fad89:::obj337:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3e4a000 2026-02-11T23:31:17.731+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc6fad89:::obj337:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.731+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc6fad89:::obj337:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.731+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.731+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.731+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.731+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc6fad89:::obj337:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.731+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.731+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.731+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:dc7d10b2:::obj533:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.731+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:dc7d10b2:::obj533:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:dc7d10b2:::obj533:head 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:dc7d10b2:::obj533:head v 107'533 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.732+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc7d10b2:::obj533:head, version: 107'533, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc7d10b2:::obj533:head@107'533, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.732+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.732+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc6fad89:::obj337:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.732+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc6fad89:::obj337:head) v3 prio 15 cost 0 latency 0.000080 pg_backfill(progress 4.0 e 115/115 lb 4:dc6fad89:::obj337:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.732+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.732+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:dc6fad89:::obj337:head) v3 2026-02-11T23:31:17.732+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6ed3c7:::obj113:head local-lis/les=112/113 n=525 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:dc6fad89:::obj337:head) 2026-02-11T23:31:17.732+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.732+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc6fad89:::obj337:head) v3 finish latency 0.000023 2026-02-11T23:31:17.732+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.732+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.732+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562bb00 con 0x3e4a000 2026-02-11T23:31:17.732+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:dc7d10b2:::obj533:head)]) -- 0x562b800 con 0x3e4a000 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.732+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1101 ==== MOSDPGPush(4.0 115/112 [PushOp(4:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51821a0 con 0x3e4a000 2026-02-11T23:31:17.732+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.732+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.732+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1102 ==== pg_backfill(progress 4.0 e 115/115 lb 4:dc7d10b2:::obj533:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3e4a000 2026-02-11T23:31:17.732+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc7d10b2:::obj533:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.732+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.732+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc7d10b2:::obj533:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.732+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.732+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc7d10b2:::obj533:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:dc9e4e8c:::obj567:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:dc9e4e8c:::obj567:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:dc9e4e8c:::obj567:head 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:dc9e4e8c:::obj567:head v 107'567 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.732+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dc9e4e8c:::obj567:head, version: 107'567, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dc9e4e8c:::obj567:head@107'567, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.732+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.732+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc7d10b2:::obj533:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.732+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc7d10b2:::obj533:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:dc7d10b2:::obj533:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.732+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.732+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:dc7d10b2:::obj533:head) v3 2026-02-11T23:31:17.732+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc6fad89:::obj337:head local-lis/les=112/113 n=526 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:dc7d10b2:::obj533:head) 2026-02-11T23:31:17.732+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.732+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc7d10b2:::obj533:head) v3 finish latency 0.000017 2026-02-11T23:31:17.732+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.732+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.732+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.732+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562b500 con 0x3e4a000 2026-02-11T23:31:17.732+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:dc9e4e8c:::obj567:head)]) -- 0x562bc80 con 0x3e4a000 2026-02-11T23:31:17.732+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.733+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1103 ==== MOSDPGPush(4.0 115/112 [PushOp(4:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3e4a000 2026-02-11T23:31:17.733+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.733+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.733+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1104 ==== pg_backfill(progress 4.0 e 115/115 lb 4:dc9e4e8c:::obj567:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded080 con 0x3e4a000 2026-02-11T23:31:17.733+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc9e4e8c:::obj567:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.733+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc9e4e8c:::obj567:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc9e4e8c:::obj567:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:dcf61aca:::obj332:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:dcf61aca:::obj332:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:dcf61aca:::obj332:head 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:dcf61aca:::obj332:head v 107'332 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.733+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dcf61aca:::obj332:head, version: 107'332, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dcf61aca:::obj332:head@107'332, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dc9e4e8c:::obj567:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc9e4e8c:::obj567:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 115/115 lb 4:dc9e4e8c:::obj567:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:dc9e4e8c:::obj567:head) v3 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc7d10b2:::obj533:head local-lis/les=112/113 n=527 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:dc9e4e8c:::obj567:head) 2026-02-11T23:31:17.733+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dc9e4e8c:::obj567:head) v3 finish latency 0.000023 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.733+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57f0600 con 0x3e4a000 2026-02-11T23:31:17.733+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:dcf61aca:::obj332:head)]) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.733+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1105 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3e4a000 2026-02-11T23:31:17.733+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.733+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.733+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1106 ==== pg_backfill(progress 4.0 e 115/115 lb 4:dcf61aca:::obj332:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3e4a000 2026-02-11T23:31:17.733+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:dcf61aca:::obj332:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.733+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dcf61aca:::obj332:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dcf61aca:::obj332:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ddce93c9:::obj164:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ddce93c9:::obj164:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ddce93c9:::obj164:head 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ddce93c9:::obj164:head v 107'164 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.733+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ddce93c9:::obj164:head, version: 107'164, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ddce93c9:::obj164:head@107'164, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dcf61aca:::obj332:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dcf61aca:::obj332:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 115/115 lb 4:dcf61aca:::obj332:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:dcf61aca:::obj332:head) v3 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dc9e4e8c:::obj567:head local-lis/les=112/113 n=528 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:dcf61aca:::obj332:head) 2026-02-11T23:31:17.733+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.733+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dcf61aca:::obj332:head) v3 finish latency 0.000018 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.733+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.733+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.733+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562af00 con 0x3e4a000 2026-02-11T23:31:17.733+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ddce93c9:::obj164:head)]) -- 0x562b080 con 0x3e4a000 2026-02-11T23:31:17.733+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.734+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1107 ==== MOSDPGPush(4.0 115/112 [PushOp(4:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.734+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.734+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.734+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.734+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1108 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ddce93c9:::obj164:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3e4a000 2026-02-11T23:31:17.734+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ddce93c9:::obj164:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.734+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ddce93c9:::obj164:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.734+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.734+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.734+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.734+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.734+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ddce93c9:::obj164:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.734+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.734+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.734+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:de0491ba:::obj374:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.734+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:de0491ba:::obj374:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.734+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:de0491ba:::obj374:head 2026-02-11T23:31:17.734+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:de0491ba:::obj374:head v 107'374 2026-02-11T23:31:17.734+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.734+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.734+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:de0491ba:::obj374:head, version: 107'374, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:de0491ba:::obj374:head@107'374, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.734+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ddce93c9:::obj164:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.734+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ddce93c9:::obj164:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:ddce93c9:::obj164:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.734+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.734+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ddce93c9:::obj164:head) v3 2026-02-11T23:31:17.734+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dcf61aca:::obj332:head local-lis/les=112/113 n=529 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ddce93c9:::obj164:head) 2026-02-11T23:31:17.734+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.734+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ddce93c9:::obj164:head) v3 finish latency 0.000015 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.734+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.734+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.734+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x57df080 con 0x3e4a000 2026-02-11T23:31:17.734+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:de0491ba:::obj374:head)]) -- 0x5610000 con 0x3e4a000 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.734+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1109 ==== MOSDPGPush(4.0 115/112 [PushOp(4:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3a00 con 0x3e4a000 2026-02-11T23:31:17.734+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.734+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.734+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1110 ==== pg_backfill(progress 4.0 e 115/115 lb 4:de0491ba:::obj374:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3e4a000 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.734+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:de0491ba:::obj374:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.734+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:de0491ba:::obj374:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.734+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.734+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.734+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.734+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.734+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:de0491ba:::obj374:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.734+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.734+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.734+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:deffd1ec:::obj134:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:deffd1ec:::obj134:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.735+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:deffd1ec:::obj134:head 2026-02-11T23:31:17.735+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:deffd1ec:::obj134:head v 107'134 2026-02-11T23:31:17.735+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.735+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.735+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:deffd1ec:::obj134:head, version: 107'134, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:deffd1ec:::obj134:head@107'134, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000080 2026-02-11T23:31:17.735+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.735+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:de0491ba:::obj374:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:de0491ba:::obj374:head) v3 prio 15 cost 0 latency 0.000097 pg_backfill(progress 4.0 e 115/115 lb 4:de0491ba:::obj374:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:de0491ba:::obj374:head) v3 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ddce93c9:::obj164:head local-lis/les=112/113 n=530 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:de0491ba:::obj374:head) 2026-02-11T23:31:17.735+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:de0491ba:::obj374:head) v3 finish latency 0.000018 2026-02-11T23:31:17.735+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.735+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.735+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.735+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562ac00 con 0x3e4a000 2026-02-11T23:31:17.735+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:deffd1ec:::obj134:head)]) -- 0x562ad80 con 0x3e4a000 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.735+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1111 ==== MOSDPGPush(4.0 115/112 [PushOp(4:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.735+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.735+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.735+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.735+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1112 ==== pg_backfill(progress 4.0 e 115/115 lb 4:deffd1ec:::obj134:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3e4a000 2026-02-11T23:31:17.735+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:deffd1ec:::obj134:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.735+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.735+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:deffd1ec:::obj134:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.735+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.735+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:deffd1ec:::obj134:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:dfa74a43:::obj27:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.735+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:dfa74a43:::obj27:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:dfa74a43:::obj27:head 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:dfa74a43:::obj27:head v 107'27 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.735+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.735+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:dfa74a43:::obj27:head, version: 107'27, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:dfa74a43:::obj27:head@107'27, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.735+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:deffd1ec:::obj134:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.735+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:deffd1ec:::obj134:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 115/115 lb 4:deffd1ec:::obj134:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.735+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.735+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:deffd1ec:::obj134:head) v3 2026-02-11T23:31:17.735+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:de0491ba:::obj374:head local-lis/les=112/113 n=531 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:deffd1ec:::obj134:head) 2026-02-11T23:31:17.735+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.735+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:deffd1ec:::obj134:head) v3 finish latency 0.000024 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.735+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.735+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.735+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4c1e480 con 0x3e4a000 2026-02-11T23:31:17.735+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:dfa74a43:::obj27:head)]) -- 0x5610180 con 0x3e4a000 2026-02-11T23:31:17.735+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.736+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1113 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.736+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.736+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.736+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1114 ==== pg_backfill(progress 4.0 e 115/115 lb 4:dfa74a43:::obj27:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3e4a000 2026-02-11T23:31:17.736+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:dfa74a43:::obj27:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.736+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dfa74a43:::obj27:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dfa74a43:::obj27:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e03f6950:::obj575:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e03f6950:::obj575:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e03f6950:::obj575:head 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e03f6950:::obj575:head v 107'575 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.736+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e03f6950:::obj575:head, version: 107'575, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e03f6950:::obj575:head@107'575, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:dfa74a43:::obj27:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dfa74a43:::obj27:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:dfa74a43:::obj27:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:dfa74a43:::obj27:head) v3 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:deffd1ec:::obj134:head local-lis/les=112/113 n=532 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:dfa74a43:::obj27:head) 2026-02-11T23:31:17.736+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:dfa74a43:::obj27:head) v3 finish latency 0.000012 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.736+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562aa80 con 0x3e4a000 2026-02-11T23:31:17.736+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e03f6950:::obj575:head)]) -- 0x5611980 con 0x3e4a000 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.736+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1115 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2000 con 0x3e4a000 2026-02-11T23:31:17.736+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.736+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.736+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1116 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e03f6950:::obj575:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3e4a000 2026-02-11T23:31:17.736+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e03f6950:::obj575:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.736+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e03f6950:::obj575:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e03f6950:::obj575:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e05d79f1:::obj206:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e05d79f1:::obj206:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e05d79f1:::obj206:head 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e05d79f1:::obj206:head v 107'206 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.736+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e05d79f1:::obj206:head, version: 107'206, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e05d79f1:::obj206:head@107'206, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e03f6950:::obj575:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e03f6950:::obj575:head) v3 prio 15 cost 0 latency 0.000079 pg_backfill(progress 4.0 e 115/115 lb 4:e03f6950:::obj575:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e03f6950:::obj575:head) v3 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:dfa74a43:::obj27:head local-lis/les=112/113 n=533 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e03f6950:::obj575:head) 2026-02-11T23:31:17.736+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.736+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e03f6950:::obj575:head) v3 finish latency 0.000018 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.736+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.736+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.736+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562a180 con 0x3e4a000 2026-02-11T23:31:17.736+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e05d79f1:::obj206:head)]) -- 0x562a600 con 0x3e4a000 2026-02-11T23:31:17.736+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.737+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1117 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a24e0 con 0x3e4a000 2026-02-11T23:31:17.737+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.737+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.737+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1118 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e05d79f1:::obj206:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3e4a000 2026-02-11T23:31:17.737+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e05d79f1:::obj206:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.737+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e05d79f1:::obj206:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e05d79f1:::obj206:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e0a44829:::obj14:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e0a44829:::obj14:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e0a44829:::obj14:head 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e0a44829:::obj14:head v 107'14 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.737+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e0a44829:::obj14:head, version: 107'14, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e0a44829:::obj14:head@107'14, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e05d79f1:::obj206:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.737+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e05d79f1:::obj206:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:e05d79f1:::obj206:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.737+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.737+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e05d79f1:::obj206:head) v3 2026-02-11T23:31:17.737+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e03f6950:::obj575:head local-lis/les=112/113 n=534 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e05d79f1:::obj206:head) 2026-02-11T23:31:17.737+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.737+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e05d79f1:::obj206:head) v3 finish latency 0.000013 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.737+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.737+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562a480 con 0x3e4a000 2026-02-11T23:31:17.737+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e0a44829:::obj14:head)]) -- 0x55dea80 con 0x3e4a000 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.737+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1119 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a29c0 con 0x3e4a000 2026-02-11T23:31:17.737+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.737+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.737+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1120 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e0a44829:::obj14:head) ==== 1061+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3e4a000 2026-02-11T23:31:17.737+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e0a44829:::obj14:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.737+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e0a44829:::obj14:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e0a44829:::obj14:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e10533c6:::obj450:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e10533c6:::obj450:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e10533c6:::obj450:head 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e10533c6:::obj450:head v 107'450 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.737+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.737+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e10533c6:::obj450:head, version: 107'450, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e10533c6:::obj450:head@107'450, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.737+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e0a44829:::obj14:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.737+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e0a44829:::obj14:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:e0a44829:::obj14:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.737+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.737+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e0a44829:::obj14:head) v3 2026-02-11T23:31:17.737+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e05d79f1:::obj206:head local-lis/les=112/113 n=535 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e0a44829:::obj14:head) 2026-02-11T23:31:17.737+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.737+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e0a44829:::obj14:head) v3 finish latency 0.000010 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.737+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.738+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.738+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.738+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7e00 con 0x3e4a000 2026-02-11T23:31:17.738+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e10533c6:::obj450:head)]) -- 0x5611800 con 0x3e4a000 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.738+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1121 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2ea0 con 0x3e4a000 2026-02-11T23:31:17.738+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.738+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.738+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.738+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1122 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e10533c6:::obj450:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3e4a000 2026-02-11T23:31:17.738+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e10533c6:::obj450:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.738+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.738+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e10533c6:::obj450:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.738+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.738+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e10533c6:::obj450:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.738+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.738+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.738+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.738+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.738+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e196274b:::obj304:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.738+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e196274b:::obj304:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.738+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e196274b:::obj304:head 2026-02-11T23:31:17.738+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e196274b:::obj304:head v 107'304 2026-02-11T23:31:17.738+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.738+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.738+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e196274b:::obj304:head, version: 107'304, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e196274b:::obj304:head@107'304, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000059 2026-02-11T23:31:17.738+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e10533c6:::obj450:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.738+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e10533c6:::obj450:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 115/115 lb 4:e10533c6:::obj450:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.738+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.738+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e10533c6:::obj450:head) v3 2026-02-11T23:31:17.738+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e0a44829:::obj14:head local-lis/les=112/113 n=536 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e10533c6:::obj450:head) 2026-02-11T23:31:17.738+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.738+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e10533c6:::obj450:head) v3 finish latency 0.000010 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.738+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.738+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.738+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562a000 con 0x3e4a000 2026-02-11T23:31:17.738+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e196274b:::obj304:head)]) -- 0x5611680 con 0x3e4a000 2026-02-11T23:31:17.738+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.739+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1123 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f2000 con 0x3e4a000 2026-02-11T23:31:17.739+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.739+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.739+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1124 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e196274b:::obj304:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3e4a000 2026-02-11T23:31:17.739+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e196274b:::obj304:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.739+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e196274b:::obj304:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e196274b:::obj304:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e1d56785:::obj141:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e1d56785:::obj141:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e1d56785:::obj141:head 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e1d56785:::obj141:head v 107'141 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.739+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e1d56785:::obj141:head, version: 107'141, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e1d56785:::obj141:head@107'141, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e196274b:::obj304:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e196274b:::obj304:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:e196274b:::obj304:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e196274b:::obj304:head) v3 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e10533c6:::obj450:head local-lis/les=112/113 n=537 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e196274b:::obj304:head) 2026-02-11T23:31:17.739+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e196274b:::obj304:head) v3 finish latency 0.000010 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.739+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7980 con 0x3e4a000 2026-02-11T23:31:17.739+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e1d56785:::obj141:head)]) -- 0x5611200 con 0x3e4a000 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.739+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1125 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3860 con 0x3e4a000 2026-02-11T23:31:17.739+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.739+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.739+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1126 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e1d56785:::obj141:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5080 con 0x3e4a000 2026-02-11T23:31:17.739+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e1d56785:::obj141:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.739+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e1d56785:::obj141:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e1d56785:::obj141:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e25e0189:::obj579:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e25e0189:::obj579:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e25e0189:::obj579:head 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e25e0189:::obj579:head v 107'579 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.739+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e25e0189:::obj579:head, version: 107'579, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e25e0189:::obj579:head@107'579, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e1d56785:::obj141:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e1d56785:::obj141:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 115/115 lb 4:e1d56785:::obj141:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e1d56785:::obj141:head) v3 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e196274b:::obj304:head local-lis/les=112/113 n=538 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e1d56785:::obj141:head) 2026-02-11T23:31:17.739+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.739+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e1d56785:::obj141:head) v3 finish latency 0.000017 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.739+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.739+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.739+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6f00 con 0x3e4a000 2026-02-11T23:31:17.739+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e25e0189:::obj579:head)]) -- 0x55f7b00 con 0x3e4a000 2026-02-11T23:31:17.739+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.740+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1127 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183ba0 con 0x3e4a000 2026-02-11T23:31:17.740+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.740+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.740+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1128 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e25e0189:::obj579:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3e4a000 2026-02-11T23:31:17.740+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e25e0189:::obj579:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.740+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e25e0189:::obj579:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e25e0189:::obj579:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e26151b8:::obj492:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e26151b8:::obj492:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e26151b8:::obj492:head 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e26151b8:::obj492:head v 107'492 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.740+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e26151b8:::obj492:head, version: 107'492, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e26151b8:::obj492:head@107'492, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000085 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e25e0189:::obj579:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.740+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e25e0189:::obj579:head) v3 prio 15 cost 0 latency 0.000101 pg_backfill(progress 4.0 e 115/115 lb 4:e25e0189:::obj579:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.740+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.740+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e25e0189:::obj579:head) v3 2026-02-11T23:31:17.740+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e1d56785:::obj141:head local-lis/les=112/113 n=539 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e25e0189:::obj579:head) 2026-02-11T23:31:17.740+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.740+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e25e0189:::obj579:head) v3 finish latency 0.000018 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.740+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.740+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0f00 con 0x3e4a000 2026-02-11T23:31:17.740+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e26151b8:::obj492:head)]) -- 0x5610d80 con 0x3e4a000 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.740+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1129 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.740+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.740+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e3cd6c93:::obj259:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e3cd6c93:::obj259:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e3cd6c93:::obj259:head 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e3cd6c93:::obj259:head v 107'259 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.740+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.740+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1130 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e26151b8:::obj492:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.740+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e26151b8:::obj492:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.740+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e26151b8:::obj492:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e26151b8:::obj492:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.740+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e3cd6c93:::obj259:head, version: 107'259, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e3cd6c93:::obj259:head@107'259, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.740+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e26151b8:::obj492:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.740+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e26151b8:::obj492:head) v3 prio 15 cost 0 latency 0.000023 pg_backfill(progress 4.0 e 115/115 lb 4:e26151b8:::obj492:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.740+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.740+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e26151b8:::obj492:head) v3 2026-02-11T23:31:17.740+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e25e0189:::obj579:head local-lis/les=112/113 n=540 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e26151b8:::obj492:head) 2026-02-11T23:31:17.740+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.740+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e26151b8:::obj492:head) v3 finish latency 0.000020 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.740+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.741+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.741+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.741+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7200 con 0x3e4a000 2026-02-11T23:31:17.741+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e3cd6c93:::obj259:head)]) -- 0x5610900 con 0x3e4a000 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.741+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1131 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.741+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 115 2026-02-11T23:31:17.741+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.741+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.741+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1132 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e3cd6c93:::obj259:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5600 con 0x3e4a000 2026-02-11T23:31:17.741+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e3cd6c93:::obj259:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.741+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e3cd6c93:::obj259:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.741+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.741+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.741+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.741+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000030 MOSDPGPush(4.0 115/112 [PushOp(4:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.741+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e3cd6c93:::obj259:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.741+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.741+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.741+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e4109927:::obj482:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.741+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e4109927:::obj482:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.741+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e4109927:::obj482:head 2026-02-11T23:31:17.741+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e4109927:::obj482:head v 107'482 2026-02-11T23:31:17.741+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.741+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.741+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e4109927:::obj482:head, version: 107'482, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e4109927:::obj482:head@107'482, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.741+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e3cd6c93:::obj259:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.741+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e3cd6c93:::obj259:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 115/115 lb 4:e3cd6c93:::obj259:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.741+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.741+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e3cd6c93:::obj259:head) v3 2026-02-11T23:31:17.741+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e26151b8:::obj492:head local-lis/les=112/113 n=541 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e3cd6c93:::obj259:head) 2026-02-11T23:31:17.741+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.741+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e3cd6c93:::obj259:head) v3 finish latency 0.000013 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.741+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.741+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.741+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55dfc80 con 0x3e4a000 2026-02-11T23:31:17.741+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e4109927:::obj482:head)]) -- 0x4bf0f00 con 0x3e4a000 2026-02-11T23:31:17.741+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.741+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1133 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.742+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000010 epoch 115 2026-02-11T23:31:17.742+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.742+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1134 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e4109927:::obj482:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.742+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e4109927:::obj482:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.742+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e4109927:::obj482:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e4109927:::obj482:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e43ffde6:::obj285:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e43ffde6:::obj285:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e43ffde6:::obj285:head 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e43ffde6:::obj285:head v 107'285 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.742+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e43ffde6:::obj285:head, version: 107'285, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e43ffde6:::obj285:head@107'285, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e4109927:::obj482:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e4109927:::obj482:head) v3 prio 15 cost 0 latency 0.000057 pg_backfill(progress 4.0 e 115/115 lb 4:e4109927:::obj482:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e4109927:::obj482:head) v3 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e3cd6c93:::obj259:head local-lis/les=112/113 n=542 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e4109927:::obj482:head) 2026-02-11T23:31:17.742+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e4109927:::obj482:head) v3 finish latency 0.000019 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.742+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6900 con 0x3e4a000 2026-02-11T23:31:17.742+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e43ffde6:::obj285:head)]) -- 0x55f6f00 con 0x3e4a000 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.742+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1135 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.742+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000009 epoch 115 2026-02-11T23:31:17.742+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.742+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1136 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e43ffde6:::obj285:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.742+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e43ffde6:::obj285:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.742+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e43ffde6:::obj285:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e43ffde6:::obj285:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e46d208d:::obj89:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e46d208d:::obj89:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e46d208d:::obj89:head 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e46d208d:::obj89:head v 107'89 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.742+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.742+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e46d208d:::obj89:head, version: 107'89, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e46d208d:::obj89:head@107'89, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e43ffde6:::obj285:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e43ffde6:::obj285:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:e43ffde6:::obj285:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e43ffde6:::obj285:head) v3 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e4109927:::obj482:head local-lis/les=112/113 n=543 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e43ffde6:::obj285:head) 2026-02-11T23:31:17.742+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e43ffde6:::obj285:head) v3 finish latency 0.000014 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.742+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.742+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.742+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55df200 con 0x3e4a000 2026-02-11T23:31:17.742+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e46d208d:::obj89:head)]) -- 0x55f7980 con 0x3e4a000 2026-02-11T23:31:17.742+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.743+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1137 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3e4a000 2026-02-11T23:31:17.743+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.743+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.743+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1138 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e46d208d:::obj89:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.743+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e46d208d:::obj89:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.743+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e46d208d:::obj89:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e46d208d:::obj89:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e48b6d56:::obj389:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e48b6d56:::obj389:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e48b6d56:::obj389:head 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e48b6d56:::obj389:head v 107'389 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.743+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e48b6d56:::obj389:head, version: 107'389, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e48b6d56:::obj389:head@107'389, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e46d208d:::obj89:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.743+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e46d208d:::obj89:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:e46d208d:::obj89:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.743+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.743+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e46d208d:::obj89:head) v3 2026-02-11T23:31:17.743+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e43ffde6:::obj285:head local-lis/les=112/113 n=544 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e46d208d:::obj89:head) 2026-02-11T23:31:17.743+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.743+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e46d208d:::obj89:head) v3 finish latency 0.000020 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.743+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.743+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6c00 con 0x3e4a000 2026-02-11T23:31:17.743+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e48b6d56:::obj389:head)]) -- 0x5611680 con 0x3e4a000 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.743+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1139 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea1a0 con 0x3e4a000 2026-02-11T23:31:17.743+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.743+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.743+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1140 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e48b6d56:::obj389:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.743+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e48b6d56:::obj389:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.743+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e48b6d56:::obj389:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e48b6d56:::obj389:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e5124c98:::obj133:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e5124c98:::obj133:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e5124c98:::obj133:head 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e5124c98:::obj133:head v 107'133 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.743+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.743+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e5124c98:::obj133:head, version: 107'133, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e5124c98:::obj133:head@107'133, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.743+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e48b6d56:::obj389:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.743+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e48b6d56:::obj389:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 115/115 lb 4:e48b6d56:::obj389:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.743+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.743+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e48b6d56:::obj389:head) v3 2026-02-11T23:31:17.743+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e46d208d:::obj89:head local-lis/les=112/113 n=545 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e48b6d56:::obj389:head) 2026-02-11T23:31:17.743+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.743+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e48b6d56:::obj389:head) v3 finish latency 0.000013 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.743+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.744+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.744+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.744+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55df680 con 0x3e4a000 2026-02-11T23:31:17.744+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e5124c98:::obj133:head)]) -- 0x5611800 con 0x3e4a000 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.744+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1141 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3e4a000 2026-02-11T23:31:17.744+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.744+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.744+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.744+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1142 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e5124c98:::obj133:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.744+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e5124c98:::obj133:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.744+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e5124c98:::obj133:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.744+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.744+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.744+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.744+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.744+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e5124c98:::obj133:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.744+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.744+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.744+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e6508d05:::obj595:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.744+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e6508d05:::obj595:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.744+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e6508d05:::obj595:head 2026-02-11T23:31:17.744+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e6508d05:::obj595:head v 107'595 2026-02-11T23:31:17.744+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.744+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.744+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e6508d05:::obj595:head, version: 107'595, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6508d05:::obj595:head@107'595, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.744+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e5124c98:::obj133:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.744+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e5124c98:::obj133:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:e5124c98:::obj133:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.744+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.744+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e5124c98:::obj133:head) v3 2026-02-11T23:31:17.744+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e48b6d56:::obj389:head local-lis/les=112/113 n=546 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e5124c98:::obj133:head) 2026-02-11T23:31:17.744+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.744+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e5124c98:::obj133:head) v3 finish latency 0.000018 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.744+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.744+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.744+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ba2d80 con 0x3e4a000 2026-02-11T23:31:17.744+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e6508d05:::obj595:head)]) -- 0x55dea80 con 0x3e4a000 2026-02-11T23:31:17.744+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.744+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1143 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x516dd40 con 0x3e4a000 2026-02-11T23:31:17.744+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.744+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.745+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1144 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e6508d05:::obj595:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.745+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e6508d05:::obj595:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.745+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e6508d05:::obj595:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000032 MOSDPGPush(4.0 115/112 [PushOp(4:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e6508d05:::obj595:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e6e28dcd:::obj87:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e6e28dcd:::obj87:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e6e28dcd:::obj87:head 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e6e28dcd:::obj87:head v 107'87 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.745+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e6e28dcd:::obj87:head, version: 107'87, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e6e28dcd:::obj87:head@107'87, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000063 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e6508d05:::obj595:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e6508d05:::obj595:head) v3 prio 15 cost 0 latency 0.000084 pg_backfill(progress 4.0 e 115/115 lb 4:e6508d05:::obj595:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e6508d05:::obj595:head) v3 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e5124c98:::obj133:head local-lis/les=112/113 n=547 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e6508d05:::obj595:head) 2026-02-11T23:31:17.745+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e6508d05:::obj595:head) v3 finish latency 0.000017 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.745+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6180 con 0x3e4a000 2026-02-11T23:31:17.745+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e6e28dcd:::obj87:head)]) -- 0x4bf0a80 con 0x3e4a000 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.745+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1145 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183380 con 0x3e4a000 2026-02-11T23:31:17.745+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.745+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.745+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1146 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e6e28dcd:::obj87:head) ==== 1061+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.745+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e6e28dcd:::obj87:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.745+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e6e28dcd:::obj87:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e6e28dcd:::obj87:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e74a4ee3:::obj396:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e74a4ee3:::obj396:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e74a4ee3:::obj396:head 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e74a4ee3:::obj396:head v 107'396 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.745+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.745+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e74a4ee3:::obj396:head, version: 107'396, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e74a4ee3:::obj396:head@107'396, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e6e28dcd:::obj87:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e6e28dcd:::obj87:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:e6e28dcd:::obj87:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e6e28dcd:::obj87:head) v3 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6508d05:::obj595:head local-lis/les=112/113 n=548 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e6e28dcd:::obj87:head) 2026-02-11T23:31:17.745+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e6e28dcd:::obj87:head) v3 finish latency 0.000018 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.745+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.745+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.745+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.745+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e74a4ee3:::obj396:head)]) -- 0x562a600 con 0x3e4a000 2026-02-11T23:31:17.745+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.746+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1147 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de44e0 con 0x3e4a000 2026-02-11T23:31:17.746+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.746+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.746+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1148 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e74a4ee3:::obj396:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164100 con 0x3e4a000 2026-02-11T23:31:17.746+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e74a4ee3:::obj396:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.746+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e74a4ee3:::obj396:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000028 MOSDPGPush(4.0 115/112 [PushOp(4:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e74a4ee3:::obj396:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e7b9c028:::obj510:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e7b9c028:::obj510:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e7b9c028:::obj510:head 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e7b9c028:::obj510:head v 107'510 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.746+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e7b9c028:::obj510:head, version: 107'510, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7b9c028:::obj510:head@107'510, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e74a4ee3:::obj396:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.746+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e74a4ee3:::obj396:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 115/115 lb 4:e74a4ee3:::obj396:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.746+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.746+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e74a4ee3:::obj396:head) v3 2026-02-11T23:31:17.746+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e6e28dcd:::obj87:head local-lis/les=112/113 n=549 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e74a4ee3:::obj396:head) 2026-02-11T23:31:17.746+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.746+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e74a4ee3:::obj396:head) v3 finish latency 0.000014 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.746+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.746+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564a000 con 0x3e4a000 2026-02-11T23:31:17.746+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e7b9c028:::obj510:head)]) -- 0x5611980 con 0x3e4a000 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.746+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1149 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d040 con 0x3e4a000 2026-02-11T23:31:17.746+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.746+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.746+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1150 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e7b9c028:::obj510:head) ==== 1062+0+0 (unknown 0 0 0) 0x4def700 con 0x3e4a000 2026-02-11T23:31:17.746+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e7b9c028:::obj510:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.746+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e7b9c028:::obj510:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e7b9c028:::obj510:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e7f87738:::obj356:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e7f87738:::obj356:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e7f87738:::obj356:head 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e7f87738:::obj356:head v 107'356 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.746+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.746+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e7f87738:::obj356:head, version: 107'356, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e7f87738:::obj356:head@107'356, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.746+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e7b9c028:::obj510:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.746+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e7b9c028:::obj510:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:e7b9c028:::obj510:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.746+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.746+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e7b9c028:::obj510:head) v3 2026-02-11T23:31:17.746+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e74a4ee3:::obj396:head local-lis/les=112/113 n=550 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e7b9c028:::obj510:head) 2026-02-11T23:31:17.746+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.746+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e7b9c028:::obj510:head) v3 finish latency 0.000018 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.746+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.747+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.747+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.747+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564a180 con 0x3e4a000 2026-02-11T23:31:17.747+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e7f87738:::obj356:head)]) -- 0x5610180 con 0x3e4a000 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.747+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1151 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.747+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.747+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.747+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.747+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1152 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e7f87738:::obj356:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dedb80 con 0x3e4a000 2026-02-11T23:31:17.747+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e7f87738:::obj356:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.747+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.747+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e7f87738:::obj356:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.747+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.747+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e7f87738:::obj356:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.747+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.747+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.747+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.747+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.747+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e8366861:::obj250:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.747+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e8366861:::obj250:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.747+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e8366861:::obj250:head 2026-02-11T23:31:17.747+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e8366861:::obj250:head v 107'250 2026-02-11T23:31:17.747+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.747+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.747+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e8366861:::obj250:head, version: 107'250, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e8366861:::obj250:head@107'250, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000058 2026-02-11T23:31:17.747+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e7f87738:::obj356:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.747+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e7f87738:::obj356:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 115/115 lb 4:e7f87738:::obj356:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.747+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.747+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e7f87738:::obj356:head) v3 2026-02-11T23:31:17.747+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7b9c028:::obj510:head local-lis/les=112/113 n=551 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e7f87738:::obj356:head) 2026-02-11T23:31:17.747+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.747+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e7f87738:::obj356:head) v3 finish latency 0.000018 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.747+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.747+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.747+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564a300 con 0x3e4a000 2026-02-11T23:31:17.747+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e8366861:::obj250:head)]) -- 0x562ad80 con 0x3e4a000 2026-02-11T23:31:17.747+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.748+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1153 ==== MOSDPGPush(4.0 115/112 [PushOp(4:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.748+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.748+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.748+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1154 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e8366861:::obj250:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bc100 con 0x3e4a000 2026-02-11T23:31:17.748+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e8366861:::obj250:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.748+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e8366861:::obj250:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e8366861:::obj250:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:e9ac1484:::obj417:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:e9ac1484:::obj417:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:e9ac1484:::obj417:head 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:e9ac1484:::obj417:head v 107'417 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.748+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:e9ac1484:::obj417:head, version: 107'417, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:e9ac1484:::obj417:head@107'417, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e8366861:::obj250:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e8366861:::obj250:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:e8366861:::obj250:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e8366861:::obj250:head) v3 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e7f87738:::obj356:head local-lis/les=112/113 n=552 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e8366861:::obj250:head) 2026-02-11T23:31:17.748+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e8366861:::obj250:head) v3 finish latency 0.000012 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.748+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564a480 con 0x3e4a000 2026-02-11T23:31:17.748+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:e9ac1484:::obj417:head)]) -- 0x5610000 con 0x3e4a000 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.748+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1155 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.748+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.748+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.748+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1156 ==== pg_backfill(progress 4.0 e 115/115 lb 4:e9ac1484:::obj417:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6ec00 con 0x3e4a000 2026-02-11T23:31:17.748+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:e9ac1484:::obj417:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 115 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.748+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e9ac1484:::obj417:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e9ac1484:::obj417:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ea485012:::obj312:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ea485012:::obj312:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ea485012:::obj312:head 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ea485012:::obj312:head v 107'312 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.748+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ea485012:::obj312:head, version: 107'312, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea485012:::obj312:head@107'312, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:e9ac1484:::obj417:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e9ac1484:::obj417:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:e9ac1484:::obj417:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:e9ac1484:::obj417:head) v3 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e8366861:::obj250:head local-lis/les=112/113 n=553 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:e9ac1484:::obj417:head) 2026-02-11T23:31:17.748+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.748+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:e9ac1484:::obj417:head) v3 finish latency 0.000017 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.748+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.748+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.748+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564a900 con 0x3e4a000 2026-02-11T23:31:17.748+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ea485012:::obj312:head)]) -- 0x564a600 con 0x3e4a000 2026-02-11T23:31:17.748+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.749+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1157 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.749+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.749+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.749+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1158 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ea485012:::obj312:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c000 con 0x3e4a000 2026-02-11T23:31:17.749+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ea485012:::obj312:head) v3 prio 15 type 95 cost 0 latency 0.000004 epoch 115 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.749+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ea485012:::obj312:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ea485012:::obj312:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ea54a43e:::obj502:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ea54a43e:::obj502:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ea54a43e:::obj502:head 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ea54a43e:::obj502:head v 107'502 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.749+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ea54a43e:::obj502:head, version: 107'502, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ea54a43e:::obj502:head@107'502, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ea485012:::obj312:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ea485012:::obj312:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:ea485012:::obj312:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ea485012:::obj312:head) v3 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:e9ac1484:::obj417:head local-lis/les=112/113 n=554 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ea485012:::obj312:head) 2026-02-11T23:31:17.749+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ea485012:::obj312:head) v3 finish latency 0.000015 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.749+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564ad80 con 0x3e4a000 2026-02-11T23:31:17.749+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ea54a43e:::obj502:head)]) -- 0x564aa80 con 0x3e4a000 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.749+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1159 ==== MOSDPGPush(4.0 115/112 [PushOp(4:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ebba0 con 0x3e4a000 2026-02-11T23:31:17.749+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.749+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.749+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1160 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ea54a43e:::obj502:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6c580 con 0x3e4a000 2026-02-11T23:31:17.749+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ea54a43e:::obj502:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.749+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ea54a43e:::obj502:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ea54a43e:::obj502:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:eaf65c1c:::obj380:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:eaf65c1c:::obj380:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:eaf65c1c:::obj380:head 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:eaf65c1c:::obj380:head v 107'380 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.749+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.749+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eaf65c1c:::obj380:head, version: 107'380, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eaf65c1c:::obj380:head@107'380, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ea54a43e:::obj502:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ea54a43e:::obj502:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:ea54a43e:::obj502:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ea54a43e:::obj502:head) v3 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea485012:::obj312:head local-lis/les=112/113 n=555 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ea54a43e:::obj502:head) 2026-02-11T23:31:17.749+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ea54a43e:::obj502:head) v3 finish latency 0.000018 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.749+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.749+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.749+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564af00 con 0x3e4a000 2026-02-11T23:31:17.749+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:eaf65c1c:::obj380:head)]) -- 0x562b080 con 0x3e4a000 2026-02-11T23:31:17.749+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.750+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1161 ==== MOSDPGPush(4.0 115/112 [PushOp(4:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a36c0 con 0x3e4a000 2026-02-11T23:31:17.750+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.750+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.750+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1162 ==== pg_backfill(progress 4.0 e 115/115 lb 4:eaf65c1c:::obj380:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6cb00 con 0x3e4a000 2026-02-11T23:31:17.750+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:eaf65c1c:::obj380:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.750+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eaf65c1c:::obj380:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eaf65c1c:::obj380:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:eb0c167c:::obj188:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:eb0c167c:::obj188:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:eb0c167c:::obj188:head 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:eb0c167c:::obj188:head v 107'188 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.750+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eb0c167c:::obj188:head, version: 107'188, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb0c167c:::obj188:head@107'188, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eaf65c1c:::obj380:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.750+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eaf65c1c:::obj380:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:eaf65c1c:::obj380:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.750+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.750+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:eaf65c1c:::obj380:head) v3 2026-02-11T23:31:17.750+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ea54a43e:::obj502:head local-lis/les=112/113 n=556 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:eaf65c1c:::obj380:head) 2026-02-11T23:31:17.750+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.750+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eaf65c1c:::obj380:head) v3 finish latency 0.000015 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.750+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.750+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564b080 con 0x3e4a000 2026-02-11T23:31:17.750+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:eb0c167c:::obj188:head)]) -- 0x4c97080 con 0x3e4a000 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.750+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1163 ==== MOSDPGPush(4.0 115/112 [PushOp(4:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f21a0 con 0x3e4a000 2026-02-11T23:31:17.750+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000008 epoch 115 2026-02-11T23:31:17.750+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.750+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1164 ==== pg_backfill(progress 4.0 e 115/115 lb 4:eb0c167c:::obj188:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6d600 con 0x3e4a000 2026-02-11T23:31:17.750+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:eb0c167c:::obj188:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.750+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eb0c167c:::obj188:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eb0c167c:::obj188:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:eb29d295:::obj554:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:eb29d295:::obj554:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:eb29d295:::obj554:head 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:eb29d295:::obj554:head v 107'554 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.750+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.750+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eb29d295:::obj554:head, version: 107'554, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb29d295:::obj554:head@107'554, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.750+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eb0c167c:::obj188:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.750+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eb0c167c:::obj188:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:eb0c167c:::obj188:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.750+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.750+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:eb0c167c:::obj188:head) v3 2026-02-11T23:31:17.750+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eaf65c1c:::obj380:head local-lis/les=112/113 n=557 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:eb0c167c:::obj188:head) 2026-02-11T23:31:17.750+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.750+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eb0c167c:::obj188:head) v3 finish latency 0.000014 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.750+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.751+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564b200 con 0x3e4a000 2026-02-11T23:31:17.751+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:eb29d295:::obj554:head)]) -- 0x562bc80 con 0x3e4a000 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.751+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1165 ==== MOSDPGPush(4.0 115/112 [PushOp(4:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2d00 con 0x3e4a000 2026-02-11T23:31:17.751+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.751+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.751+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1166 ==== pg_backfill(progress 4.0 e 115/115 lb 4:eb29d295:::obj554:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6db80 con 0x3e4a000 2026-02-11T23:31:17.751+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:eb29d295:::obj554:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.751+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eb29d295:::obj554:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eb29d295:::obj554:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:eb806c57:::obj373:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:eb806c57:::obj373:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:eb806c57:::obj373:head 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:eb806c57:::obj373:head v 107'373 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.751+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eb806c57:::obj373:head, version: 107'373, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eb806c57:::obj373:head@107'373, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eb29d295:::obj554:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eb29d295:::obj554:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:eb29d295:::obj554:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:eb29d295:::obj554:head) v3 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb0c167c:::obj188:head local-lis/les=112/113 n=558 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:eb29d295:::obj554:head) 2026-02-11T23:31:17.751+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eb29d295:::obj554:head) v3 finish latency 0.000017 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.751+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564b680 con 0x3e4a000 2026-02-11T23:31:17.751+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:eb806c57:::obj373:head)]) -- 0x564b380 con 0x3e4a000 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.751+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1167 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2820 con 0x3e4a000 2026-02-11T23:31:17.751+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.751+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.751+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1168 ==== pg_backfill(progress 4.0 e 115/115 lb 4:eb806c57:::obj373:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e100 con 0x3e4a000 2026-02-11T23:31:17.751+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:eb806c57:::obj373:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 115 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.751+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eb806c57:::obj373:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eb806c57:::obj373:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ecbe4ab7:::obj125:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ecbe4ab7:::obj125:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ecbe4ab7:::obj125:head 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ecbe4ab7:::obj125:head v 107'125 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.751+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.751+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ecbe4ab7:::obj125:head, version: 107'125, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ecbe4ab7:::obj125:head@107'125, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.751+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eb806c57:::obj373:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eb806c57:::obj373:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:eb806c57:::obj373:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:eb806c57:::obj373:head) v3 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb29d295:::obj554:head local-lis/les=112/113 n=559 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:eb806c57:::obj373:head) 2026-02-11T23:31:17.751+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.751+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eb806c57:::obj373:head) v3 finish latency 0.000019 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.751+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.752+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.752+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564b800 con 0x3e4a000 2026-02-11T23:31:17.752+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ecbe4ab7:::obj125:head)]) -- 0x562b800 con 0x3e4a000 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.752+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1169 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2340 con 0x3e4a000 2026-02-11T23:31:17.752+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.752+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.752+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1170 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ecbe4ab7:::obj125:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6e680 con 0x3e4a000 2026-02-11T23:31:17.752+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ecbe4ab7:::obj125:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.752+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ecbe4ab7:::obj125:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.752+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.752+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ecbe4ab7:::obj125:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ed234181:::obj472:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ed234181:::obj472:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ed234181:::obj472:head 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ed234181:::obj472:head v 107'472 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.752+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ed234181:::obj472:head, version: 107'472, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ed234181:::obj472:head@107'472, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.752+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.752+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ecbe4ab7:::obj125:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.752+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ecbe4ab7:::obj125:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:ecbe4ab7:::obj125:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.752+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.752+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ecbe4ab7:::obj125:head) v3 2026-02-11T23:31:17.752+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eb806c57:::obj373:head local-lis/les=112/113 n=560 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ecbe4ab7:::obj125:head) 2026-02-11T23:31:17.752+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.752+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ecbe4ab7:::obj125:head) v3 finish latency 0.000017 2026-02-11T23:31:17.752+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.752+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.752+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564bc80 con 0x3e4a000 2026-02-11T23:31:17.752+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ed234181:::obj472:head)]) -- 0x564b980 con 0x3e4a000 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.752+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1171 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43eba00 con 0x3e4a000 2026-02-11T23:31:17.752+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.752+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.752+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1172 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ed234181:::obj472:head) ==== 1062+0+0 (unknown 0 0 0) 0x4e6f700 con 0x3e4a000 2026-02-11T23:31:17.752+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ed234181:::obj472:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.752+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ed234181:::obj472:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.752+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.752+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ed234181:::obj472:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ee44f25c:::obj334:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.752+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ee44f25c:::obj334:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ee44f25c:::obj334:head 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ee44f25c:::obj334:head v 107'334 2026-02-11T23:31:17.752+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.752+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.753+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ee44f25c:::obj334:head, version: 107'334, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ee44f25c:::obj334:head@107'334, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000082 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ed234181:::obj472:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ed234181:::obj472:head) v3 prio 15 cost 0 latency 0.000102 pg_backfill(progress 4.0 e 115/115 lb 4:ed234181:::obj472:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ed234181:::obj472:head) v3 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ecbe4ab7:::obj125:head local-lis/les=112/113 n=561 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ed234181:::obj472:head) 2026-02-11T23:31:17.753+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ed234181:::obj472:head) v3 finish latency 0.000026 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.753+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.753+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564aa80 con 0x3e4a000 2026-02-11T23:31:17.753+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ee44f25c:::obj334:head)]) -- 0x564be00 con 0x3e4a000 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.753+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1173 ==== MOSDPGPush(4.0 115/112 [PushOp(4:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4e7f6c0 con 0x3e4a000 2026-02-11T23:31:17.753+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.753+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.753+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1174 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ee44f25c:::obj334:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5cc00 con 0x3e4a000 2026-02-11T23:31:17.753+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ee44f25c:::obj334:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.753+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ee44f25c:::obj334:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ee44f25c:::obj334:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:eeae2a94:::obj11:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:eeae2a94:::obj11:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:eeae2a94:::obj11:head 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:eeae2a94:::obj11:head v 107'11 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.753+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.753+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eeae2a94:::obj11:head, version: 107'11, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eeae2a94:::obj11:head@107'11, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ee44f25c:::obj334:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.753+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ee44f25c:::obj334:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:ee44f25c:::obj334:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.753+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.753+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ee44f25c:::obj334:head) v3 2026-02-11T23:31:17.753+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ed234181:::obj472:head local-lis/les=112/113 n=562 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ee44f25c:::obj334:head) 2026-02-11T23:31:17.753+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.753+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ee44f25c:::obj334:head) v3 finish latency 0.000020 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.753+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.753+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.753+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564ad80 con 0x3e4a000 2026-02-11T23:31:17.753+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:eeae2a94:::obj11:head)]) -- 0x57f1b00 con 0x3e4a000 2026-02-11T23:31:17.753+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.754+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1175 ==== MOSDPGPush(4.0 115/112 [PushOp(4:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3ba0 con 0x3e4a000 2026-02-11T23:31:17.754+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.754+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.754+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1176 ==== pg_backfill(progress 4.0 e 115/115 lb 4:eeae2a94:::obj11:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5bb80 con 0x3e4a000 2026-02-11T23:31:17.754+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:eeae2a94:::obj11:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.754+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eeae2a94:::obj11:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eeae2a94:::obj11:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:eff0a739:::obj127:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:eff0a739:::obj127:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:eff0a739:::obj127:head 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:eff0a739:::obj127:head v 107'127 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.754+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:eff0a739:::obj127:head, version: 107'127, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:eff0a739:::obj127:head@107'127, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000060 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eeae2a94:::obj11:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eeae2a94:::obj11:head) v3 prio 15 cost 0 latency 0.000075 pg_backfill(progress 4.0 e 115/115 lb 4:eeae2a94:::obj11:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:eeae2a94:::obj11:head) v3 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ee44f25c:::obj334:head local-lis/les=112/113 n=563 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:eeae2a94:::obj11:head) 2026-02-11T23:31:17.754+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eeae2a94:::obj11:head) v3 finish latency 0.000016 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.754+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564a600 con 0x3e4a000 2026-02-11T23:31:17.754+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:eff0a739:::obj127:head)]) -- 0x562b680 con 0x3e4a000 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.754+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1177 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b040 con 0x3e4a000 2026-02-11T23:31:17.754+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.754+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.754+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1178 ==== pg_backfill(progress 4.0 e 115/115 lb 4:eff0a739:::obj127:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5a000 con 0x3e4a000 2026-02-11T23:31:17.754+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:eff0a739:::obj127:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.754+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eff0a739:::obj127:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000029 MOSDPGPush(4.0 115/112 [PushOp(4:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eff0a739:::obj127:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f0a12295:::obj487:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f0a12295:::obj487:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f0a12295:::obj487:head 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f0a12295:::obj487:head v 107'487 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.754+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.754+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f0a12295:::obj487:head, version: 107'487, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f0a12295:::obj487:head@107'487, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000062 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:eff0a739:::obj127:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eff0a739:::obj127:head) v3 prio 15 cost 0 latency 0.000085 pg_backfill(progress 4.0 e 115/115 lb 4:eff0a739:::obj127:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:eff0a739:::obj127:head) v3 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eeae2a94:::obj11:head local-lis/les=112/113 n=564 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:eff0a739:::obj127:head) 2026-02-11T23:31:17.754+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:eff0a739:::obj127:head) v3 finish latency 0.000026 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.754+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.754+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.754+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.754+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564a900 con 0x3e4a000 2026-02-11T23:31:17.755+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f0a12295:::obj487:head)]) -- 0x5658000 con 0x3e4a000 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.755+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1179 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434d040 con 0x3e4a000 2026-02-11T23:31:17.755+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.755+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.755+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.755+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1180 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f0a12295:::obj487:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5ab00 con 0x3e4a000 2026-02-11T23:31:17.755+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f0a12295:::obj487:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.755+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f0a12295:::obj487:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.755+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.755+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.755+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.755+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.755+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f0a12295:::obj487:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.755+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.755+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.755+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f1182822:::obj193:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.755+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f1182822:::obj193:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.755+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f1182822:::obj193:head 2026-02-11T23:31:17.755+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f1182822:::obj193:head v 107'193 2026-02-11T23:31:17.755+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.755+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.755+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f1182822:::obj193:head, version: 107'193, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f1182822:::obj193:head@107'193, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000057 2026-02-11T23:31:17.755+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f0a12295:::obj487:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f0a12295:::obj487:head) v3 prio 15 cost 0 latency 0.000076 pg_backfill(progress 4.0 e 115/115 lb 4:f0a12295:::obj487:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f0a12295:::obj487:head) v3 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:eff0a739:::obj127:head local-lis/les=112/113 n=565 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f0a12295:::obj487:head) 2026-02-11T23:31:17.755+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f0a12295:::obj487:head) v3 finish latency 0.000016 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.755+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.755+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5610000 con 0x3e4a000 2026-02-11T23:31:17.755+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f1182822:::obj193:head)]) -- 0x5658300 con 0x3e4a000 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.755+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1181 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x4de44e0 con 0x3e4a000 2026-02-11T23:31:17.755+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.755+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.755+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1182 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f1182822:::obj193:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5d180 con 0x3e4a000 2026-02-11T23:31:17.755+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f1182822:::obj193:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.755+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f1182822:::obj193:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.755+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000027 MOSDPGPush(4.0 115/112 [PushOp(4:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.755+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f1182822:::obj193:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f11aef48:::obj58:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.755+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f11aef48:::obj58:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f11aef48:::obj58:head 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f11aef48:::obj58:head v 107'58 2026-02-11T23:31:17.755+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.755+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.756+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f11aef48:::obj58:head, version: 107'58, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f11aef48:::obj58:head@107'58, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000064 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.756+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.756+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f1182822:::obj193:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f1182822:::obj193:head) v3 prio 15 cost 0 latency 0.000087 pg_backfill(progress 4.0 e 115/115 lb 4:f1182822:::obj193:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f1182822:::obj193:head) v3 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f0a12295:::obj487:head local-lis/les=112/113 n=566 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f1182822:::obj193:head) 2026-02-11T23:31:17.756+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f1182822:::obj193:head) v3 finish latency 0.000017 2026-02-11T23:31:17.756+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.756+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.756+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.756+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562ad80 con 0x3e4a000 2026-02-11T23:31:17.756+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f11aef48:::obj58:head)]) -- 0x564a480 con 0x3e4a000 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.756+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1183 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183380 con 0x3e4a000 2026-02-11T23:31:17.756+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.756+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.756+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.756+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1184 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f11aef48:::obj58:head) ==== 1061+0+0 (unknown 0 0 0) 0x4f5a580 con 0x3e4a000 2026-02-11T23:31:17.756+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f11aef48:::obj58:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.756+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.756+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f11aef48:::obj58:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.756+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.756+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f11aef48:::obj58:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f13c731e:::obj449:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.756+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f13c731e:::obj449:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f13c731e:::obj449:head 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f13c731e:::obj449:head v 107'449 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.756+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.756+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f13c731e:::obj449:head, version: 107'449, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f13c731e:::obj449:head@107'449, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.756+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f11aef48:::obj58:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.756+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f11aef48:::obj58:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:f11aef48:::obj58:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.756+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.756+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f11aef48:::obj58:head) v3 2026-02-11T23:31:17.756+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f1182822:::obj193:head local-lis/les=112/113 n=567 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f11aef48:::obj58:head) 2026-02-11T23:31:17.756+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.756+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f11aef48:::obj58:head) v3 finish latency 0.000015 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.756+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.756+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.756+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564a300 con 0x3e4a000 2026-02-11T23:31:17.756+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f13c731e:::obj449:head)]) -- 0x5658600 con 0x3e4a000 2026-02-11T23:31:17.756+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.757+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1185 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dd40 con 0x3e4a000 2026-02-11T23:31:17.757+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.757+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.757+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1186 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f13c731e:::obj449:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162000 con 0x3e4a000 2026-02-11T23:31:17.757+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f13c731e:::obj449:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 115 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.757+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f13c731e:::obj449:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f13c731e:::obj449:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f22dcb20:::obj569:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f22dcb20:::obj569:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f22dcb20:::obj569:head 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f22dcb20:::obj569:head v 107'569 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.757+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f22dcb20:::obj569:head, version: 107'569, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f22dcb20:::obj569:head@107'569, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f13c731e:::obj449:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f13c731e:::obj449:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:f13c731e:::obj449:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f13c731e:::obj449:head) v3 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f11aef48:::obj58:head local-lis/les=112/113 n=568 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f13c731e:::obj449:head) 2026-02-11T23:31:17.757+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f13c731e:::obj449:head) v3 finish latency 0.000010 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.757+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5610180 con 0x3e4a000 2026-02-11T23:31:17.757+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f22dcb20:::obj569:head)]) -- 0x5658900 con 0x3e4a000 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.757+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1187 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5182340 con 0x3e4a000 2026-02-11T23:31:17.757+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.757+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.757+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1188 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f22dcb20:::obj569:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162580 con 0x3e4a000 2026-02-11T23:31:17.757+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f22dcb20:::obj569:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.757+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f22dcb20:::obj569:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f22dcb20:::obj569:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f2a1ba99:::obj364:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f2a1ba99:::obj364:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f2a1ba99:::obj364:head 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f2a1ba99:::obj364:head v 107'364 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.757+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.757+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2a1ba99:::obj364:head, version: 107'364, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2a1ba99:::obj364:head@107'364, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f22dcb20:::obj569:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f22dcb20:::obj569:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:f22dcb20:::obj569:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f22dcb20:::obj569:head) v3 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f13c731e:::obj449:head local-lis/les=112/113 n=569 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f22dcb20:::obj569:head) 2026-02-11T23:31:17.757+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f22dcb20:::obj569:head) v3 finish latency 0.000011 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.757+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.757+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.757+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564a180 con 0x3e4a000 2026-02-11T23:31:17.757+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f2a1ba99:::obj364:head)]) -- 0x5658c00 con 0x3e4a000 2026-02-11T23:31:17.757+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.758+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1189 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x43ea1a0 con 0x3e4a000 2026-02-11T23:31:17.758+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.758+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.758+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1190 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f2a1ba99:::obj364:head) ==== 1062+0+0 (unknown 0 0 0) 0x5162b00 con 0x3e4a000 2026-02-11T23:31:17.758+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2a1ba99:::obj364:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.758+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2a1ba99:::obj364:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2a1ba99:::obj364:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f2bc7f0e:::obj94:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f2bc7f0e:::obj94:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f2bc7f0e:::obj94:head 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f2bc7f0e:::obj94:head v 107'94 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.758+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2bc7f0e:::obj94:head, version: 107'94, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2bc7f0e:::obj94:head@107'94, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000056 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2a1ba99:::obj364:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2a1ba99:::obj364:head) v3 prio 15 cost 0 latency 0.000078 pg_backfill(progress 4.0 e 115/115 lb 4:f2a1ba99:::obj364:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f2a1ba99:::obj364:head) v3 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f22dcb20:::obj569:head local-lis/les=112/113 n=570 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f2a1ba99:::obj364:head) 2026-02-11T23:31:17.758+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2a1ba99:::obj364:head) v3 finish latency 0.000012 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.758+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564a000 con 0x3e4a000 2026-02-11T23:31:17.758+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f2bc7f0e:::obj94:head)]) -- 0x5611980 con 0x3e4a000 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.758+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1191 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x434c1a0 con 0x3e4a000 2026-02-11T23:31:17.758+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.758+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.758+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1192 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f2bc7f0e:::obj94:head) ==== 1061+0+0 (unknown 0 0 0) 0x5163080 con 0x3e4a000 2026-02-11T23:31:17.758+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2bc7f0e:::obj94:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.758+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2bc7f0e:::obj94:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2bc7f0e:::obj94:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f2f9dd5a:::obj121:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f2f9dd5a:::obj121:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f2f9dd5a:::obj121:head 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f2f9dd5a:::obj121:head v 107'121 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.758+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.758+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2f9dd5a:::obj121:head, version: 107'121, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2f9dd5a:::obj121:head@107'121, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2bc7f0e:::obj94:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2bc7f0e:::obj94:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:f2bc7f0e:::obj94:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f2bc7f0e:::obj94:head) v3 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2a1ba99:::obj364:head local-lis/les=112/113 n=571 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f2bc7f0e:::obj94:head) 2026-02-11T23:31:17.758+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2bc7f0e:::obj94:head) v3 finish latency 0.000015 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.758+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.758+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.758+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x562a600 con 0x3e4a000 2026-02-11T23:31:17.758+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f2f9dd5a:::obj121:head)]) -- 0x5658f00 con 0x3e4a000 2026-02-11T23:31:17.758+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.759+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1193 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dcbba0 con 0x3e4a000 2026-02-11T23:31:17.759+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.759+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.759+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1194 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f2f9dd5a:::obj121:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163600 con 0x3e4a000 2026-02-11T23:31:17.759+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2f9dd5a:::obj121:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.759+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2f9dd5a:::obj121:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2f9dd5a:::obj121:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f2fc2104:::obj429:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f2fc2104:::obj429:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f2fc2104:::obj429:head 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f2fc2104:::obj429:head v 107'429 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.759+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f2fc2104:::obj429:head, version: 107'429, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f2fc2104:::obj429:head@107'429, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2f9dd5a:::obj121:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2f9dd5a:::obj121:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:f2f9dd5a:::obj121:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f2f9dd5a:::obj121:head) v3 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2bc7f0e:::obj94:head local-lis/les=112/113 n=572 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f2f9dd5a:::obj121:head) 2026-02-11T23:31:17.759+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2f9dd5a:::obj121:head) v3 finish latency 0.000014 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.759+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f6180 con 0x3e4a000 2026-02-11T23:31:17.759+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f2fc2104:::obj429:head)]) -- 0x4ae4d80 con 0x3e4a000 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.759+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1195 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d520 con 0x3e4a000 2026-02-11T23:31:17.759+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.759+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.759+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1196 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f2fc2104:::obj429:head) ==== 1062+0+0 (unknown 0 0 0) 0x5163b80 con 0x3e4a000 2026-02-11T23:31:17.759+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2fc2104:::obj429:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.759+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2fc2104:::obj429:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2fc2104:::obj429:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f31cde5e:::obj446:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f31cde5e:::obj446:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f31cde5e:::obj446:head 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f31cde5e:::obj446:head v 107'446 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.759+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.759+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f31cde5e:::obj446:head, version: 107'446, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f31cde5e:::obj446:head@107'446, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.759+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f2fc2104:::obj429:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2fc2104:::obj429:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:f2fc2104:::obj429:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f2fc2104:::obj429:head) v3 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2f9dd5a:::obj121:head local-lis/les=112/113 n=573 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f2fc2104:::obj429:head) 2026-02-11T23:31:17.759+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.759+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f2fc2104:::obj429:head) v3 finish latency 0.000015 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.759+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.760+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55dea80 con 0x3e4a000 2026-02-11T23:31:17.760+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f31cde5e:::obj446:head)]) -- 0x5659200 con 0x3e4a000 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.760+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1197 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4de4340 con 0x3e4a000 2026-02-11T23:31:17.760+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000007 epoch 115 2026-02-11T23:31:17.760+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.760+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1198 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f31cde5e:::obj446:head) ==== 1062+0+0 (unknown 0 0 0) 0x5164680 con 0x3e4a000 2026-02-11T23:31:17.760+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f31cde5e:::obj446:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.760+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f31cde5e:::obj446:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f31cde5e:::obj446:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f33597d0:::obj489:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f33597d0:::obj489:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f33597d0:::obj489:head 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f33597d0:::obj489:head v 107'489 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.760+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f33597d0:::obj489:head, version: 107'489, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f33597d0:::obj489:head@107'489, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f31cde5e:::obj446:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f31cde5e:::obj446:head) v3 prio 15 cost 0 latency 0.000072 pg_backfill(progress 4.0 e 115/115 lb 4:f31cde5e:::obj446:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f31cde5e:::obj446:head) v3 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f2fc2104:::obj429:head local-lis/les=112/113 n=574 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f31cde5e:::obj446:head) 2026-02-11T23:31:17.760+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f31cde5e:::obj446:head) v3 finish latency 0.000016 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.760+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55df680 con 0x3e4a000 2026-02-11T23:31:17.760+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f33597d0:::obj489:head)]) -- 0x4ba2d80 con 0x3e4a000 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.760+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1199 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x39204e0 con 0x3e4a000 2026-02-11T23:31:17.760+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.760+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.760+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1200 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f33597d0:::obj489:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbf700 con 0x3e4a000 2026-02-11T23:31:17.760+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f33597d0:::obj489:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.760+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f33597d0:::obj489:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000026 MOSDPGPush(4.0 115/112 [PushOp(4:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f33597d0:::obj489:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f3768836:::obj189:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f3768836:::obj189:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f3768836:::obj189:head 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f3768836:::obj189:head v 107'189 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.760+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.760+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f3768836:::obj189:head, version: 107'189, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3768836:::obj189:head@107'189, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.760+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f33597d0:::obj489:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f33597d0:::obj489:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:f33597d0:::obj489:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f33597d0:::obj489:head) v3 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f31cde5e:::obj446:head local-lis/les=112/113 n=575 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f33597d0:::obj489:head) 2026-02-11T23:31:17.760+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.760+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f33597d0:::obj489:head) v3 finish latency 0.000016 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.760+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.761+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.761+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.761+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5611680 con 0x3e4a000 2026-02-11T23:31:17.761+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f3768836:::obj189:head)]) -- 0x5659500 con 0x3e4a000 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.761+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1201 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51a21a0 con 0x3e4a000 2026-02-11T23:31:17.761+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.761+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.761+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1202 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f3768836:::obj189:head) ==== 1062+0+0 (unknown 0 0 0) 0x48bab00 con 0x3e4a000 2026-02-11T23:31:17.761+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f3768836:::obj189:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.761+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f3768836:::obj189:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.761+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.761+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.761+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.761+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f3768836:::obj189:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.761+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.761+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.761+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f3c2593a:::obj26:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f3c2593a:::obj26:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.761+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f3c2593a:::obj26:head 2026-02-11T23:31:17.761+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f3c2593a:::obj26:head v 107'26 2026-02-11T23:31:17.761+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.761+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.761+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f3c2593a:::obj26:head, version: 107'26, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f3c2593a:::obj26:head@107'26, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.761+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.761+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f3768836:::obj189:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.761+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f3768836:::obj189:head) v3 prio 15 cost 0 latency 0.000073 pg_backfill(progress 4.0 e 115/115 lb 4:f3768836:::obj189:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.761+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.761+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f3768836:::obj189:head) v3 2026-02-11T23:31:17.761+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f33597d0:::obj489:head local-lis/les=112/113 n=576 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f3768836:::obj189:head) 2026-02-11T23:31:17.761+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.761+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f3768836:::obj189:head) v3 finish latency 0.000014 2026-02-11T23:31:17.761+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.761+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.761+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.761+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55f7980 con 0x3e4a000 2026-02-11T23:31:17.761+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f3c2593a:::obj26:head)]) -- 0x55f6c00 con 0x3e4a000 2026-02-11T23:31:17.761+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.762+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1203 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2680 con 0x3e4a000 2026-02-11T23:31:17.762+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.762+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.762+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1204 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f3c2593a:::obj26:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f6100 con 0x3e4a000 2026-02-11T23:31:17.762+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f3c2593a:::obj26:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.762+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f3c2593a:::obj26:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f3c2593a:::obj26:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f42cbfe2:::obj212:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f42cbfe2:::obj212:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f42cbfe2:::obj212:head 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f42cbfe2:::obj212:head v 107'212 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.762+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f42cbfe2:::obj212:head, version: 107'212, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f42cbfe2:::obj212:head@107'212, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f3c2593a:::obj26:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f3c2593a:::obj26:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 115/115 lb 4:f3c2593a:::obj26:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f3c2593a:::obj26:head) v3 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3768836:::obj189:head local-lis/les=112/113 n=577 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f3c2593a:::obj26:head) 2026-02-11T23:31:17.762+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f3c2593a:::obj26:head) v3 finish latency 0.000015 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.762+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55df200 con 0x3e4a000 2026-02-11T23:31:17.762+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f42cbfe2:::obj212:head)]) -- 0x5659800 con 0x3e4a000 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.762+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1205 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a2b60 con 0x3e4a000 2026-02-11T23:31:17.762+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.762+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.762+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1206 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f42cbfe2:::obj212:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6680 con 0x3e4a000 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.762+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f42cbfe2:::obj212:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.762+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f42cbfe2:::obj212:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f42cbfe2:::obj212:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f4af07f1:::obj515:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f4af07f1:::obj515:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f4af07f1:::obj515:head 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f4af07f1:::obj515:head v 107'515 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.762+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f4af07f1:::obj515:head, version: 107'515, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4af07f1:::obj515:head@107'515, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f42cbfe2:::obj212:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f42cbfe2:::obj212:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 115/115 lb 4:f42cbfe2:::obj212:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f42cbfe2:::obj212:head) v3 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f3c2593a:::obj26:head local-lis/les=112/113 n=578 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f42cbfe2:::obj212:head) 2026-02-11T23:31:17.762+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.762+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f42cbfe2:::obj212:head) v3 finish latency 0.000012 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.762+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.762+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.762+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x4bf0f00 con 0x3e4a000 2026-02-11T23:31:17.762+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f4af07f1:::obj515:head)]) -- 0x55f6f00 con 0x3e4a000 2026-02-11T23:31:17.762+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.763+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1207 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x4dfe1a0 con 0x3e4a000 2026-02-11T23:31:17.763+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.763+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.763+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1208 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f4af07f1:::obj515:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f6c00 con 0x3e4a000 2026-02-11T23:31:17.763+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f4af07f1:::obj515:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.763+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f4af07f1:::obj515:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f4af07f1:::obj515:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f4b32ae1:::obj130:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f4b32ae1:::obj130:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f4b32ae1:::obj130:head 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f4b32ae1:::obj130:head v 107'130 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.763+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f4b32ae1:::obj130:head, version: 107'130, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f4b32ae1:::obj130:head@107'130, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f4af07f1:::obj515:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f4af07f1:::obj515:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:f4af07f1:::obj515:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f4af07f1:::obj515:head) v3 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f42cbfe2:::obj212:head local-lis/les=112/113 n=579 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f4af07f1:::obj515:head) 2026-02-11T23:31:17.763+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f4af07f1:::obj515:head) v3 finish latency 0.000014 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.763+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x55dfc80 con 0x3e4a000 2026-02-11T23:31:17.763+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f4b32ae1:::obj130:head)]) -- 0x5659b00 con 0x3e4a000 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.763+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1209 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3520 con 0x3e4a000 2026-02-11T23:31:17.763+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.763+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.763+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1210 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f4b32ae1:::obj130:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f7180 con 0x3e4a000 2026-02-11T23:31:17.763+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f4b32ae1:::obj130:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 115 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.763+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f4b32ae1:::obj130:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f4b32ae1:::obj130:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f677cfaa:::obj144:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f677cfaa:::obj144:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f677cfaa:::obj144:head 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f677cfaa:::obj144:head v 107'144 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.763+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.763+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f677cfaa:::obj144:head, version: 107'144, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f677cfaa:::obj144:head@107'144, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000047 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f4b32ae1:::obj130:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f4b32ae1:::obj130:head) v3 prio 15 cost 0 latency 0.000063 pg_backfill(progress 4.0 e 115/115 lb 4:f4b32ae1:::obj130:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f4b32ae1:::obj130:head) v3 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4af07f1:::obj515:head local-lis/les=112/113 n=580 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f4b32ae1:::obj130:head) 2026-02-11T23:31:17.763+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f4b32ae1:::obj130:head) v3 finish latency 0.000011 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.763+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.763+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.763+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5610900 con 0x3e4a000 2026-02-11T23:31:17.763+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f677cfaa:::obj144:head)]) -- 0x5659e00 con 0x3e4a000 2026-02-11T23:31:17.763+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.764+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1211 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183d40 con 0x3e4a000 2026-02-11T23:31:17.764+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.764+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.764+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1212 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f677cfaa:::obj144:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5600 con 0x3e4a000 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.764+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f677cfaa:::obj144:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.764+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f677cfaa:::obj144:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f677cfaa:::obj144:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f76d37c8:::obj371:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f76d37c8:::obj371:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f76d37c8:::obj371:head 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f76d37c8:::obj371:head v 107'371 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.764+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f76d37c8:::obj371:head, version: 107'371, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f76d37c8:::obj371:head@107'371, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f677cfaa:::obj144:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f677cfaa:::obj144:head) v3 prio 15 cost 0 latency 0.000061 pg_backfill(progress 4.0 e 115/115 lb 4:f677cfaa:::obj144:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f677cfaa:::obj144:head) v3 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f4b32ae1:::obj130:head local-lis/les=112/113 n=581 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f677cfaa:::obj144:head) 2026-02-11T23:31:17.764+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f677cfaa:::obj144:head) v3 finish latency 0.000016 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.764+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664000 con 0x3e4a000 2026-02-11T23:31:17.764+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f76d37c8:::obj371:head)]) -- 0x55f7200 con 0x3e4a000 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.764+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1213 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183a00 con 0x3e4a000 2026-02-11T23:31:17.764+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.764+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.764+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1214 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f76d37c8:::obj371:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f5b80 con 0x3e4a000 2026-02-11T23:31:17.764+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f76d37c8:::obj371:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.764+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f76d37c8:::obj371:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f76d37c8:::obj371:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f77ac86f:::obj198:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f77ac86f:::obj198:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f77ac86f:::obj198:head 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f77ac86f:::obj198:head v 107'198 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.764+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.764+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f77ac86f:::obj198:head, version: 107'198, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f77ac86f:::obj198:head@107'198, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f76d37c8:::obj371:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f76d37c8:::obj371:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:f76d37c8:::obj371:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f76d37c8:::obj371:head) v3 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f677cfaa:::obj144:head local-lis/les=112/113 n=582 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f76d37c8:::obj371:head) 2026-02-11T23:31:17.764+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f76d37c8:::obj371:head) v3 finish latency 0.000014 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.764+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.764+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.764+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.765+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664180 con 0x3e4a000 2026-02-11T23:31:17.765+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f77ac86f:::obj198:head)]) -- 0x5611980 con 0x3e4a000 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.765+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1215 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5003+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.765+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.765+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.765+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1216 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f77ac86f:::obj198:head) ==== 1062+0+0 (unknown 0 0 0) 0x49f4000 con 0x3e4a000 2026-02-11T23:31:17.765+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f77ac86f:::obj198:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.765+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f77ac86f:::obj198:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f77ac86f:::obj198:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f7c5b032:::obj82:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f7c5b032:::obj82:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f7c5b032:::obj82:head 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f7c5b032:::obj82:head v 107'82 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.765+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f7c5b032:::obj82:head, version: 107'82, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7c5b032:::obj82:head@107'82, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000051 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f77ac86f:::obj198:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.765+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f77ac86f:::obj198:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:f77ac86f:::obj198:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.765+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.765+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f77ac86f:::obj198:head) v3 2026-02-11T23:31:17.765+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f76d37c8:::obj371:head local-lis/les=112/113 n=583 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f77ac86f:::obj198:head) 2026-02-11T23:31:17.765+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.765+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f77ac86f:::obj198:head) v3 finish latency 0.000010 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.765+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.765+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664300 con 0x3e4a000 2026-02-11T23:31:17.765+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f7c5b032:::obj82:head)]) -- 0x5658c00 con 0x3e4a000 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.765+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1217 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.765+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.765+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.765+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1218 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f7c5b032:::obj82:head) ==== 1061+0+0 (unknown 0 0 0) 0x49f5080 con 0x3e4a000 2026-02-11T23:31:17.765+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f7c5b032:::obj82:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.765+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f7c5b032:::obj82:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f7c5b032:::obj82:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f7eaa9b8:::obj104:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f7eaa9b8:::obj104:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f7eaa9b8:::obj104:head 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f7eaa9b8:::obj104:head v 107'104 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.765+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.765+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f7eaa9b8:::obj104:head, version: 107'104, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f7eaa9b8:::obj104:head@107'104, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.765+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f7c5b032:::obj82:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.765+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f7c5b032:::obj82:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:f7c5b032:::obj82:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.765+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.765+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f7c5b032:::obj82:head) v3 2026-02-11T23:31:17.765+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f77ac86f:::obj198:head local-lis/les=112/113 n=584 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f7c5b032:::obj82:head) 2026-02-11T23:31:17.765+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.765+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f7c5b032:::obj82:head) v3 finish latency 0.000011 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.765+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.766+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.766+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664480 con 0x3e4a000 2026-02-11T23:31:17.766+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f7eaa9b8:::obj104:head)]) -- 0x564a180 con 0x3e4a000 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.766+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1219 ==== MOSDPGPush(4.0 115/112 [PushOp(4:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43f3a00 con 0x3e4a000 2026-02-11T23:31:17.766+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.766+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.766+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1220 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f7eaa9b8:::obj104:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c100 con 0x3e4a000 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.766+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f7eaa9b8:::obj104:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.766+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f7eaa9b8:::obj104:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f7eaa9b8:::obj104:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:f948044a:::obj145:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:f948044a:::obj145:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:f948044a:::obj145:head 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:f948044a:::obj145:head v 107'145 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.766+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:f948044a:::obj145:head, version: 107'145, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:f948044a:::obj145:head@107'145, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f7eaa9b8:::obj104:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.766+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f7eaa9b8:::obj104:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:f7eaa9b8:::obj104:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.766+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.766+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f7eaa9b8:::obj104:head) v3 2026-02-11T23:31:17.766+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7c5b032:::obj82:head local-lis/les=112/113 n=585 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f7eaa9b8:::obj104:head) 2026-02-11T23:31:17.766+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.766+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f7eaa9b8:::obj104:head) v3 finish latency 0.000010 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.766+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.766+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664600 con 0x3e4a000 2026-02-11T23:31:17.766+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:f948044a:::obj145:head)]) -- 0x5610180 con 0x3e4a000 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.766+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1221 ==== MOSDPGPush(4.0 115/112 [PushOp(4:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.766+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.766+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.766+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1222 ==== pg_backfill(progress 4.0 e 115/115 lb 4:f948044a:::obj145:head) ==== 1062+0+0 (unknown 0 0 0) 0x4f5c680 con 0x3e4a000 2026-02-11T23:31:17.766+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:f948044a:::obj145:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.766+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f948044a:::obj145:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f948044a:::obj145:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:fa02d04b:::obj266:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:fa02d04b:::obj266:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:fa02d04b:::obj266:head 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:fa02d04b:::obj266:head v 107'266 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.766+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.766+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fa02d04b:::obj266:head, version: 107'266, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa02d04b:::obj266:head@107'266, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000048 2026-02-11T23:31:17.766+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.766+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:f948044a:::obj145:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f948044a:::obj145:head) v3 prio 15 cost 0 latency 0.000064 pg_backfill(progress 4.0 e 115/115 lb 4:f948044a:::obj145:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:f948044a:::obj145:head) v3 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f7eaa9b8:::obj104:head local-lis/les=112/113 n=586 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:f948044a:::obj145:head) 2026-02-11T23:31:17.767+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:f948044a:::obj145:head) v3 finish latency 0.000013 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.767+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.767+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664780 con 0x3e4a000 2026-02-11T23:31:17.767+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:fa02d04b:::obj266:head)]) -- 0x564a300 con 0x3e4a000 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.767+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1223 ==== MOSDPGPush(4.0 115/112 [PushOp(4:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51a3a00 con 0x3e4a000 2026-02-11T23:31:17.767+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.767+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.767+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.767+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1224 ==== pg_backfill(progress 4.0 e 115/115 lb 4:fa02d04b:::obj266:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbc580 con 0x3e4a000 2026-02-11T23:31:17.767+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:fa02d04b:::obj266:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 115 2026-02-11T23:31:17.767+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.767+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fa02d04b:::obj266:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.767+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.767+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fa02d04b:::obj266:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.767+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.767+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.767+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.767+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.767+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:fa6c2b5c:::obj438:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.767+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:fa6c2b5c:::obj438:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.767+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:fa6c2b5c:::obj438:head 2026-02-11T23:31:17.767+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:fa6c2b5c:::obj438:head v 107'438 2026-02-11T23:31:17.767+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.767+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.767+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fa6c2b5c:::obj438:head, version: 107'438, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fa6c2b5c:::obj438:head@107'438, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.767+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fa02d04b:::obj266:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fa02d04b:::obj266:head) v3 prio 15 cost 0 latency 0.000066 pg_backfill(progress 4.0 e 115/115 lb 4:fa02d04b:::obj266:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:fa02d04b:::obj266:head) v3 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:f948044a:::obj145:head local-lis/les=112/113 n=587 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:fa02d04b:::obj266:head) 2026-02-11T23:31:17.767+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fa02d04b:::obj266:head) v3 finish latency 0.000010 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.767+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.767+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.767+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664900 con 0x3e4a000 2026-02-11T23:31:17.767+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:fa6c2b5c:::obj438:head)]) -- 0x562ad80 con 0x3e4a000 2026-02-11T23:31:17.767+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.768+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1225 ==== MOSDPGPush(4.0 115/112 [PushOp(4:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x43ea000 con 0x3e4a000 2026-02-11T23:31:17.768+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.768+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.768+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1226 ==== pg_backfill(progress 4.0 e 115/115 lb 4:fa6c2b5c:::obj438:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbcb00 con 0x3e4a000 2026-02-11T23:31:17.768+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:fa6c2b5c:::obj438:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.768+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fa6c2b5c:::obj438:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fa6c2b5c:::obj438:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:fb333d49:::obj180:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:fb333d49:::obj180:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:fb333d49:::obj180:head 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:fb333d49:::obj180:head v 107'180 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.768+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fb333d49:::obj180:head, version: 107'180, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb333d49:::obj180:head@107'180, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fa6c2b5c:::obj438:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fa6c2b5c:::obj438:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:fa6c2b5c:::obj438:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:fa6c2b5c:::obj438:head) v3 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa02d04b:::obj266:head local-lis/les=112/113 n=588 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:fa6c2b5c:::obj438:head) 2026-02-11T23:31:17.768+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fa6c2b5c:::obj438:head) v3 finish latency 0.000011 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.768+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664a80 con 0x3e4a000 2026-02-11T23:31:17.768+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:fb333d49:::obj180:head)]) -- 0x5610000 con 0x3e4a000 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.768+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1227 ==== MOSDPGPush(4.0 115/112 [PushOp(4:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51821a0 con 0x3e4a000 2026-02-11T23:31:17.768+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.768+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.768+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1228 ==== pg_backfill(progress 4.0 e 115/115 lb 4:fb333d49:::obj180:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd080 con 0x3e4a000 2026-02-11T23:31:17.768+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:fb333d49:::obj180:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.768+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fb333d49:::obj180:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fb333d49:::obj180:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:fb44f156:::obj279:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:fb44f156:::obj279:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:fb44f156:::obj279:head 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:fb44f156:::obj279:head v 107'279 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.768+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.768+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fb44f156:::obj279:head, version: 107'279, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb44f156:::obj279:head@107'279, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000049 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fb333d49:::obj180:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fb333d49:::obj180:head) v3 prio 15 cost 0 latency 0.000059 pg_backfill(progress 4.0 e 115/115 lb 4:fb333d49:::obj180:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:fb333d49:::obj180:head) v3 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fa6c2b5c:::obj438:head local-lis/les=112/113 n=589 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:fb333d49:::obj180:head) 2026-02-11T23:31:17.768+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fb333d49:::obj180:head) v3 finish latency 0.000009 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.768+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.768+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.768+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664c00 con 0x3e4a000 2026-02-11T23:31:17.768+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:fb44f156:::obj279:head)]) -- 0x564a900 con 0x3e4a000 2026-02-11T23:31:17.768+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.769+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1229 ==== MOSDPGPush(4.0 115/112 [PushOp(4:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516dba0 con 0x3e4a000 2026-02-11T23:31:17.769+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.769+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.769+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1230 ==== pg_backfill(progress 4.0 e 115/115 lb 4:fb44f156:::obj279:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbd600 con 0x3e4a000 2026-02-11T23:31:17.769+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:fb44f156:::obj279:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.769+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fb44f156:::obj279:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fb44f156:::obj279:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:fb8f5c57:::obj248:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:fb8f5c57:::obj248:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:fb8f5c57:::obj248:head 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:fb8f5c57:::obj248:head v 107'248 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.769+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fb8f5c57:::obj248:head, version: 107'248, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fb8f5c57:::obj248:head@107'248, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fb44f156:::obj279:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fb44f156:::obj279:head) v3 prio 15 cost 0 latency 0.000069 pg_backfill(progress 4.0 e 115/115 lb 4:fb44f156:::obj279:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:fb44f156:::obj279:head) v3 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb333d49:::obj180:head local-lis/les=112/113 n=590 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:fb44f156:::obj279:head) 2026-02-11T23:31:17.769+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fb44f156:::obj279:head) v3 finish latency 0.000011 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.769+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664d80 con 0x3e4a000 2026-02-11T23:31:17.769+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:fb8f5c57:::obj248:head)]) -- 0x564a600 con 0x3e4a000 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.769+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1231 ==== MOSDPGPush(4.0 115/112 [PushOp(4:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51831e0 con 0x3e4a000 2026-02-11T23:31:17.769+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.769+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.769+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1232 ==== pg_backfill(progress 4.0 e 115/115 lb 4:fb8f5c57:::obj248:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbdb80 con 0x3e4a000 2026-02-11T23:31:17.769+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:fb8f5c57:::obj248:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.769+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fb8f5c57:::obj248:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fb8f5c57:::obj248:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:fcc34b3b:::obj191:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:fcc34b3b:::obj191:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:fcc34b3b:::obj191:head 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:fcc34b3b:::obj191:head v 107'191 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.769+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fcc34b3b:::obj191:head, version: 107'191, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcc34b3b:::obj191:head@107'191, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000055 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fb8f5c57:::obj248:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fb8f5c57:::obj248:head) v3 prio 15 cost 0 latency 0.000074 pg_backfill(progress 4.0 e 115/115 lb 4:fb8f5c57:::obj248:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:fb8f5c57:::obj248:head) v3 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb44f156:::obj279:head local-lis/les=112/113 n=591 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:fb8f5c57:::obj248:head) 2026-02-11T23:31:17.769+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.769+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fb8f5c57:::obj248:head) v3 finish latency 0.000014 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.769+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.769+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.769+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5665200 con 0x3e4a000 2026-02-11T23:31:17.769+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:fcc34b3b:::obj191:head)]) -- 0x5664f00 con 0x3e4a000 2026-02-11T23:31:17.769+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.770+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1233 ==== MOSDPGPush(4.0 115/112 [PushOp(4:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516d1e0 con 0x3e4a000 2026-02-11T23:31:17.770+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.770+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.770+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1234 ==== pg_backfill(progress 4.0 e 115/115 lb 4:fcc34b3b:::obj191:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe100 con 0x3e4a000 2026-02-11T23:31:17.770+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:fcc34b3b:::obj191:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.770+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fcc34b3b:::obj191:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fcc34b3b:::obj191:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:fcdeb4b1:::obj379:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:fcdeb4b1:::obj379:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:fcdeb4b1:::obj379:head 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:fcdeb4b1:::obj379:head v 107'379 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.770+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fcdeb4b1:::obj379:head, version: 107'379, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fcdeb4b1:::obj379:head@107'379, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fcc34b3b:::obj191:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fcc34b3b:::obj191:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:fcc34b3b:::obj191:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:fcc34b3b:::obj191:head) v3 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fb8f5c57:::obj248:head local-lis/les=112/113 n=592 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:fcc34b3b:::obj191:head) 2026-02-11T23:31:17.770+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fcc34b3b:::obj191:head) v3 finish latency 0.000015 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.770+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5665380 con 0x3e4a000 2026-02-11T23:31:17.770+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:fcdeb4b1:::obj379:head)]) -- 0x564ad80 con 0x3e4a000 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.770+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1235 ==== MOSDPGPush(4.0 115/112 [PushOp(4:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x516cb60 con 0x3e4a000 2026-02-11T23:31:17.770+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.770+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.770+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1236 ==== pg_backfill(progress 4.0 e 115/115 lb 4:fcdeb4b1:::obj379:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dbe680 con 0x3e4a000 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.770+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:fcdeb4b1:::obj379:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.770+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fcdeb4b1:::obj379:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fcdeb4b1:::obj379:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:fd91bcb6:::obj528:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:fd91bcb6:::obj528:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:fd91bcb6:::obj528:head 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:fd91bcb6:::obj528:head v 107'528 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.770+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.770+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fd91bcb6:::obj528:head, version: 107'528, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fd91bcb6:::obj528:head@107'528, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fcdeb4b1:::obj379:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fcdeb4b1:::obj379:head) v3 prio 15 cost 0 latency 0.000062 pg_backfill(progress 4.0 e 115/115 lb 4:fcdeb4b1:::obj379:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:fcdeb4b1:::obj379:head) v3 2026-02-11T23:31:17.770+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcc34b3b:::obj191:head local-lis/les=112/113 n=593 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:fcdeb4b1:::obj379:head) 2026-02-11T23:31:17.770+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.770+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fcdeb4b1:::obj379:head) v3 finish latency 0.000016 2026-02-11T23:31:17.770+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.771+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.771+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.771+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5665800 con 0x3e4a000 2026-02-11T23:31:17.771+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:fd91bcb6:::obj528:head)]) -- 0x5665500 con 0x3e4a000 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.771+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1237 ==== MOSDPGPush(4.0 115/112 [PushOp(4:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x43f3a00 con 0x3e4a000 2026-02-11T23:31:17.771+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.771+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.771+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.771+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1238 ==== pg_backfill(progress 4.0 e 115/115 lb 4:fd91bcb6:::obj528:head) ==== 1062+0+0 (unknown 0 0 0) 0x4decb00 con 0x3e4a000 2026-02-11T23:31:17.771+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.771+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:fd91bcb6:::obj528:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.771+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fd91bcb6:::obj528:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.771+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.771+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.771+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000024 MOSDPGPush(4.0 115/112 [PushOp(4:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.771+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fd91bcb6:::obj528:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.771+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.771+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.771+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:fe351e27:::obj9:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.771+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:fe351e27:::obj9:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.771+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:fe351e27:::obj9:head 2026-02-11T23:31:17.771+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:fe351e27:::obj9:head v 107'9 2026-02-11T23:31:17.771+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.771+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.771+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fe351e27:::obj9:head, version: 107'9, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fe351e27:::obj9:head@107'9, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000050 2026-02-11T23:31:17.771+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fd91bcb6:::obj528:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.771+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fd91bcb6:::obj528:head) v3 prio 15 cost 0 latency 0.000065 pg_backfill(progress 4.0 e 115/115 lb 4:fd91bcb6:::obj528:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.771+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.771+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:fd91bcb6:::obj528:head) v3 2026-02-11T23:31:17.771+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fcdeb4b1:::obj379:head local-lis/les=112/113 n=594 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:fd91bcb6:::obj528:head) 2026-02-11T23:31:17.771+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.771+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fd91bcb6:::obj528:head) v3 finish latency 0.000014 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.771+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.771+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.771+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5665980 con 0x3e4a000 2026-02-11T23:31:17.771+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:fe351e27:::obj9:head)]) -- 0x564aa80 con 0x3e4a000 2026-02-11T23:31:17.771+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.772+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1239 ==== MOSDPGPush(4.0 115/112 [PushOp(4:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x389b1e0 con 0x3e4a000 2026-02-11T23:31:17.772+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.772+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.772+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1240 ==== pg_backfill(progress 4.0 e 115/115 lb 4:fe351e27:::obj9:head) ==== 1060+0+0 (unknown 0 0 0) 0x4ded080 con 0x3e4a000 2026-02-11T23:31:17.772+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:fe351e27:::obj9:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.772+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fe351e27:::obj9:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fe351e27:::obj9:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:fedf1386:::obj565:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:fedf1386:::obj565:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:fedf1386:::obj565:head 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:fedf1386:::obj565:head v 107'565 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.772+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:fedf1386:::obj565:head, version: 107'565, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:fedf1386:::obj565:head@107'565, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fe351e27:::obj9:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fe351e27:::obj9:head) v3 prio 15 cost 0 latency 0.000071 pg_backfill(progress 4.0 e 115/115 lb 4:fe351e27:::obj9:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:fe351e27:::obj9:head) v3 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fd91bcb6:::obj528:head local-lis/les=112/113 n=595 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:fe351e27:::obj9:head) 2026-02-11T23:31:17.772+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fe351e27:::obj9:head) v3 finish latency 0.000014 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.772+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5665e00 con 0x3e4a000 2026-02-11T23:31:17.772+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:fedf1386:::obj565:head)]) -- 0x5665b00 con 0x3e4a000 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.772+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1241 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x51836c0 con 0x3e4a000 2026-02-11T23:31:17.772+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.772+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.772+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1242 ==== pg_backfill(progress 4.0 e 115/115 lb 4:fedf1386:::obj565:head) ==== 1062+0+0 (unknown 0 0 0) 0x4ded600 con 0x3e4a000 2026-02-11T23:31:17.772+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:fedf1386:::obj565:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.772+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fedf1386:::obj565:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000025 MOSDPGPush(4.0 115/112 [PushOp(4:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fedf1386:::obj565:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ff38ecdd:::obj155:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ff38ecdd:::obj155:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ff38ecdd:::obj155:head 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ff38ecdd:::obj155:head v 107'155 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.772+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.772+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ff38ecdd:::obj155:head, version: 107'155, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff38ecdd:::obj155:head@107'155, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000054 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:fedf1386:::obj565:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fedf1386:::obj565:head) v3 prio 15 cost 0 latency 0.000070 pg_backfill(progress 4.0 e 115/115 lb 4:fedf1386:::obj565:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:fedf1386:::obj565:head) v3 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fe351e27:::obj9:head local-lis/les=112/113 n=596 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:fedf1386:::obj565:head) 2026-02-11T23:31:17.772+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:fedf1386:::obj565:head) v3 finish latency 0.000016 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.772+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.772+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.772+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664900 con 0x3e4a000 2026-02-11T23:31:17.772+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ff38ecdd:::obj155:head)]) -- 0x564b980 con 0x3e4a000 2026-02-11T23:31:17.772+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.773+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1243 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 4999+0+0 (unknown 0 0 0) 0x5183a00 con 0x3e4a000 2026-02-11T23:31:17.773+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.773+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.773+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1244 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ff38ecdd:::obj155:head) ==== 1062+0+0 (unknown 0 0 0) 0x4dee100 con 0x3e4a000 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.773+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ff38ecdd:::obj155:head) v3 prio 15 type 95 cost 0 latency 0.000003 epoch 115 2026-02-11T23:31:17.773+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ff38ecdd:::obj155:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ff38ecdd:::obj155:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ff7b1f36:::obj1:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ff7b1f36:::obj1:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ff7b1f36:::obj1:head 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ff7b1f36:::obj1:head v 107'1 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.773+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ff7b1f36:::obj1:head, version: 107'1, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ff7b1f36:::obj1:head@107'1, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000052 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ff38ecdd:::obj155:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.773+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ff38ecdd:::obj155:head) v3 prio 15 cost 0 latency 0.000068 pg_backfill(progress 4.0 e 115/115 lb 4:ff38ecdd:::obj155:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.773+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.773+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ff38ecdd:::obj155:head) v3 2026-02-11T23:31:17.773+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:fedf1386:::obj565:head local-lis/les=112/113 n=597 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ff38ecdd:::obj155:head) 2026-02-11T23:31:17.773+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.773+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ff38ecdd:::obj155:head) v3 finish latency 0.000011 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.773+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.773+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x564a300 con 0x3e4a000 2026-02-11T23:31:17.773+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ff7b1f36:::obj1:head)]) -- 0x562b800 con 0x3e4a000 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.773+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1245 ==== MOSDPGPush(4.0 115/112 [PushOp(4:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) ==== 5007+0+0 (unknown 0 0 0) 0x5183d40 con 0x3e4a000 2026-02-11T23:31:17.773+0000 7fac703fa640 15 osd.0 115 enqueue_op MOSDPGPush(4.0 115/112 [PushOp(4:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 type 105 cost 5096 latency 0.000006 epoch 115 2026-02-11T23:31:17.773+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0 prio 15 cost 5096 e115) 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.773+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1246 ==== pg_backfill(progress 4.0 e 115/115 lb 4:ff7b1f36:::obj1:head) ==== 1060+0+0 (unknown 0 0 0) 0x4dee680 con 0x3e4a000 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) queued 2026-02-11T23:31:17.773+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(progress 4.0 e 115/115 lb 4:ff7b1f36:::obj1:head) v3 prio 15 type 95 cost 0 latency 0.000002 epoch 115 2026-02-11T23:31:17.773+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ff7b1f36:::obj1:head) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=MOSDPGPush(4.0 115/112 [PushOp(4:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4) class_id 0background_recovery prio 15 qos_cost 18636 cost 5096 e115) pg 0x4859000 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 prio 15 cost 5096 latency 0.000023 MOSDPGPush(4.0 115/112 [PushOp(4:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ff7b1f36:::obj1:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: MOSDPGPush(4.0 115/112 [PushOp(4:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_push ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false)ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false) 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data 4:ffd8ed3d:::obj316:head backfill size 4096 previous size 0 net size 4096 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] submit_push_data recovering object 4:ffd8ed3d:::obj316:head copy_subset: [0~4096] intervals_included: [0~4096] data_zeros: [] 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] on_local_recover: 4:ffd8ed3d:::obj316:head 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got got missing 4:ffd8ed3d:::obj316:head v 107'316 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recover_got last_complete now 107'600 log.complete_to at end 2026-02-11T23:31:17.773+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.773+0000 7fac51168640 10 osd.0 115 dequeue_op MOSDPGPush(4.0 115/112 [PushOp(4:ffd8ed3d:::obj316:head, version: 107'316, data_included: [0~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:ffd8ed3d:::obj316:head@107'316, size: 4096, copy_subset: [0~4096], clone_subset: {}, snapset: 0={}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))]) v4 finish latency 0.000053 2026-02-11T23:31:17.773+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(progress 4.0 e 115/115 lb 4:ff7b1f36:::obj1:head) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.773+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ff7b1f36:::obj1:head) v3 prio 15 cost 0 latency 0.000067 pg_backfill(progress 4.0 e 115/115 lb 4:ff7b1f36:::obj1:head) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.773+0000 7fac55170640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.773+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(progress 4.0 e 115/115 lb 4:ff7b1f36:::obj1:head) v3 2026-02-11T23:31:17.773+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff38ecdd:::obj155:head local-lis/les=112/113 n=598 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(progress 4.0 e 115/115 lb 4:ff7b1f36:::obj1:head) 2026-02-11T23:31:17.773+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.773+0000 7fac55170640 10 osd.0 115 dequeue_op pg_backfill(progress 4.0 e 115/115 lb 4:ff7b1f36:::obj1:head) v3 finish latency 0.000013 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff7b1f36:::obj1:head local-lis/les=112/113 n=599 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _applied_recovered_object_replica 2026-02-11T23:31:17.773+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.774+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff7b1f36:::obj1:head local-lis/les=112/113 n=599 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::recovery_committed_to version 107'600 now ondisk 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 115 get_con_osd_cluster to osd.4 from_epoch 115 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 115 get_nextmap_reserved map_reservations: {115=1} 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 115 release_map epoch: 115 2026-02-11T23:31:17.774+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_trim(4.0 107'600 e115/115) -- 0x5664780 con 0x3e4a000 2026-02-11T23:31:17.774+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- MOSDPGPushReply(4.0 115/112 [PushReplyOp(4:ffd8ed3d:::obj316:head)]) -- 0x564b800 con 0x3e4a000 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.774+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1247 ==== pg_backfill(finish 4.0 e 115/115 lb MAX) ==== 1098+0+0 (unknown 0 0 0) 0x4dbc000 con 0x3e4a000 2026-02-11T23:31:17.774+0000 7fac703fa640 15 osd.0 115 enqueue_op pg_backfill(finish 4.0 e 115/115 lb MAX) v3 prio 15 type 95 cost 0 latency 0.000005 epoch 115 2026-02-11T23:31:17.774+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(finish 4.0 e 115/115 lb MAX) v3) class_id 0 prio 15 cost 0 e115) 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(finish 4.0 e 115/115 lb MAX) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) queued 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGRecoveryMsg(op=pg_backfill(finish 4.0 e 115/115 lb MAX) v3) class_id 0background_recovery prio 15 qos_cost 18636 cost 0 e115) pg 0x4859000 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.774+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(finish 4.0 e 115/115 lb MAX) v3 prio 15 cost 0 latency 0.000017 pg_backfill(finish 4.0 e 115/115 lb MAX) v3 pg pg[4.0( v 107'600 (107'590,107'600] lb 4:ff7b1f36:::obj1:head local-lis/les=112/113 n=599 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] 2026-02-11T23:31:17.774+0000 7fac51168640 10 osd.0 115 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 115 is not newer than session's projected_epoch of 115 2026-02-11T23:31:17.774+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff7b1f36:::obj1:head local-lis/les=112/113 n=599 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] _handle_message: pg_backfill(finish 4.0 e 115/115 lb MAX) v3 2026-02-11T23:31:17.774+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] lb 4:ff7b1f36:::obj1:head local-lis/les=112/113 n=599 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_backfill pg_backfill(finish 4.0 e 115/115 lb MAX) 2026-02-11T23:31:17.774+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_backfill(finish_ack 4.0 e 115/115 lb MIN) -- 0x40ecc00 con 0x3e4a000 2026-02-11T23:31:17.774+0000 7fac51168640 15 osd.0 115 enqueue_peering_evt 4.0 epoch_sent: 115 epoch_requested: 115 RecoveryDone 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RecoveryDone) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=599 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::update_backfill_progress final 2457600 replaces local 2457600 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RecoveryDone) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:17.774+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.774+0000 7fac51168640 10 osd.0 115 dequeue_op pg_backfill(finish 4.0 e 115/115 lb MAX) v3 finish latency 0.000031 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RecoveryDone) class_id 2immediate prio 255 cost 10 e115) pg 0x4859000 2026-02-11T23:31:17.774+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 RecoveryDone 2026-02-11T23:31:17.774+0000 7fac55170640 5 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] exit Started/ReplicaActive/RepRecovering 0.383094 7 0.000029 2026-02-11T23:31:17.774+0000 7fac55170640 5 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:17.774+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.774+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1248 ==== MBackfillReserve(4.0 RELEASE e115/115) ==== 46+0+0 (unknown 0 0 0) 0x564b800 con 0x3e4a000 2026-02-11T23:31:17.774+0000 7fac703fa640 15 osd.0 115 enqueue_peering_evt 4.0 epoch_sent: 115 epoch_requested: 115 RemoteReservationCanceled 2026-02-11T23:31:17.774+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RemoteReservationCanceled) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RemoteReservationCanceled) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 RemoteReservationCanceled) class_id 2immediate prio 255 cost 10 e115) pg 0x4859000 2026-02-11T23:31:17.774+0000 7fac55170640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 RemoteReservationCanceled 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.774+0000 7fac55170640 5 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.000097 1 0.000011 2026-02-11T23:31:17.774+0000 7fac55170640 5 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:17.774+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.774+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1249 ==== pg_info2(4.0 4.0 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) e115/115) ==== 1122+0+0 (unknown 0 0 0) 0x4d50300 con 0x3e4a000 2026-02-11T23:31:17.774+0000 7fac703fa640 15 osd.0 115 enqueue_peering_evt 4.0 epoch_sent: 115 epoch_requested: 115 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) pg_lease(ru 76.338790894s ub 76.338790894s int 16.000000000s) 2026-02-11T23:31:17.774+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) pg_lease(ru 76.338790894s ub 76.338790894s int 16.000000000s)) class_id 2 prio 255 cost 10 e115) 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) pg_lease(ru 76.338790894s ub 76.338790894s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) queued 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 115 epoch_requested: 115 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) pg_lease(ru 76.338790894s ub 76.338790894s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e115) pg 0x4859000 2026-02-11T23:31:17.774+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.774+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 115 epoch_requested: 115 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) pg_lease(ru 76.338790894s ub 76.338790894s int 16.000000000s) 2026-02-11T23:31:17.774+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.774+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.796+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1250 ==== pg_query2(4.0 4.0 query(info 0'0 epoch_sent 116) e116/116) ==== 158+0+0 (unknown 0 0 0) 0x4c83a40 con 0x3e4a000 2026-02-11T23:31:17.796+0000 7fac703fa640 15 osd.0 115 enqueue_peering_evt 4.0 epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116) 2026-02-11T23:31:17.796+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116)) class_id 2 prio 255 cost 10 e116) 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116)) class_id 2immediate prio 255 cost 10 e116) queued 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116)) class_id 2immediate prio 255 cost 10 e116) pg 0x4859000 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 116, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116)) class_id 2immediate prio 255 cost 10 e116) 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.796+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.796+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1251 ==== osd_map(116..116 src has 1..116) ==== 317+0+0 (unknown 0 0 0) 0x3940fc0 con 0x3e4a000 2026-02-11T23:31:17.796+0000 7fac606e4640 20 osd.0 115 OSD::ms_dispatch: osd_map(116..116 src has 1..116) 2026-02-11T23:31:17.796+0000 7fac606e4640 20 osd.0 115 _dispatch 0x3940fc0 osd_map(116..116 src has 1..116) 2026-02-11T23:31:17.796+0000 7fac606e4640 3 osd.0 115 handle_osd_map epochs [116,116], i have 115, src has [1,116] 2026-02-11T23:31:17.796+0000 7fac606e4640 10 osd.0 115 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.796+0000 7fac606e4640 10 osd.0 115 handle_osd_map got inc map for epoch 116 2026-02-11T23:31:17.796+0000 7fac606e4640 20 osd.0 115 got_full_map 116, nothing requested 2026-02-11T23:31:17.796+0000 7fac606e4640 20 osd.0 115 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:17.796+0000 7fac606e4640 20 osd.0 115 handle_osd_map pg_num_history pg_num_history(e116 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:31:17.796+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {116={}} 2026-02-11T23:31:17.796+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:17.796+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e116 maps [1~116] lci=[5,116] tlb=1) 2026-02-11T23:31:17.796+0000 7fac5e983640 10 osd.0 115 _committed_osd_maps 116..116 2026-02-11T23:31:17.796+0000 7fac5e983640 10 osd.0 115 advance to epoch 116 (<= last 116 <= newest_map 116) 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0 116 consume_map version 116 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0 115 await_reserved_maps epoch:116 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0 115 await_reserved_maps done 0 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0 116 consume_map 116 -- publish done 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x4946160 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0 116 identify_splits_and_merges 4.0 e115 to e116 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0 116 identify_splits_and_merges 4.0 e115 to e116 pg_nums {45=1} 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x25809a0 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0 116 identify_splits_and_merges 1.0 e115 to e116 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0 116 identify_splits_and_merges 1.0 e115 to e116 pg_nums {24=1} 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0 116 prune_pg_created 2026-02-11T23:31:17.796+0000 7fac5e983640 10 osd.0:0.consume_map 116 (was 115) 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 116 <= 116, requeueing 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {116=} 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:17.796+0000 7fac5e983640 10 osd.0:1.consume_map 116 (was 115) 2026-02-11T23:31:17.796+0000 7fac5e983640 10 osd.0:2.consume_map 116 (was 115) 2026-02-11T23:31:17.796+0000 7fac5e983640 10 osd.0:3.consume_map 116 (was 115) 2026-02-11T23:31:17.796+0000 7fac5e983640 10 osd.0:4.consume_map 116 (was 115) 2026-02-11T23:31:17.796+0000 7fac5e983640 10 osd.0:5.consume_map 116 (was 115) 2026-02-11T23:31:17.796+0000 7fac5e983640 10 osd.0:6.consume_map 116 (was 115) 2026-02-11T23:31:17.796+0000 7fac5e983640 10 osd.0:7.consume_map 116 (was 115) 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116)) class_id 2immediate prio 255 cost 10 e116) queued 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116)) class_id 2immediate prio 255 cost 10 e116) pg 0x4859000 2026-02-11T23:31:17.796+0000 7fac5e983640 10 osd.0 116 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] handle_advance_map: 116 2026-02-11T23:31:17.796+0000 7fac5e983640 15 osd.0 116 enqueue_peering_evt 4.0 epoch_sent: 116 epoch_requested: 116 NullEvt 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 115 -> 116 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 NullEvt) class_id 2 prio 255 cost 10 e116) 2026-02-11T23:31:17.796+0000 7fac5e983640 15 osd.0 116 enqueue_peering_evt 1.0 epoch_sent: 116 epoch_requested: 116 NullEvt 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] state: Started advmap 2026-02-11T23:31:17.796+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 NullEvt) class_id 2 prio 255 cost 10 e116) 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::should_restart_peering new interval newup [4,0] newacting [4,0] 2026-02-11T23:31:17.796+0000 7fac5e983640 7 osd.0 116 activate_map version 116 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] state: should_restart_peering, transitioning to Reset 2026-02-11T23:31:17.796+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.796+0000 7fac51168640 5 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] exit Started/ReplicaActive/RepNotRecovering 0.021983 2 0.000004 2026-02-11T23:31:17.796+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 NullEvt) class_id 2immediate prio 255 cost 10 e116) queued 2026-02-11T23:31:17.796+0000 7fac51168640 5 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] exit Started/ReplicaActive 0.406435 0 0.000000 2026-02-11T23:31:17.796+0000 7fac51168640 5 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] exit Started 0.406670 0 0.000000 2026-02-11T23:31:17.796+0000 7fac51168640 5 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] enter Reset 2026-02-11T23:31:17.796+0000 7fac55971640 20 osd.0 116 agent_entry empty queue 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=112 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 116 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=116 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::set_last_peering_reset Clearing blocked outgoing recovery messages 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=116 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::set_last_peering_reset Not blocking outgoing recovery messages 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=116 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] state: Reset advmap 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=116 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::should_restart_peering new interval newup [4,0] newacting [4,0] 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=116 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] state: should restart peering, calling start_peering_interval again 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0]/[4] r=-1 lpr=116 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::set_last_peering_reset set_last_peering_reset 116 2026-02-11T23:31:17.796+0000 7fac51168640 15 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0] r=-1 lpr=116 pi=[106,112)/1 pct=0'0 crt=107'600 active mbc={}] clear_stats 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::start_peering_interval : check_new_interval output: check_new_interval interval(112-115 up [4, 0](4) acting [4](4)) up_thru 112 up_from 21 last_epoch_clean 107 interval(112-115 up [4, 0](4) acting [4](4) maybe_went_rw) : primary up 21-112 includes interval 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=112) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::start_peering_interval noting past ([106,115] all_participants={0, 4} intervals=[([112,115] acting={4})]) 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::on_new_interval 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::on_new_interval upacting_features 0x3f0ffffffffdffff from [4,0]+[4,0] 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::on_new_interval checking missing set deletes flag. missing = missing(0 may_include_deletes = 1) 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::init_hb_stamps now [0x3863810] 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::on_new_interval prior_readable_until_ub 0.000000000s (mnow 70.631225586s + 0.000000000s) 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] cancel_recovery 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] clear_recovery_state 2026-02-11T23:31:17.796+0000 7fac51168640 15 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] _clear_recovery_state 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:17.796+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] scrubber: FSM: ReplicaActive::exit() 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] scrubber: FSM: ReplicaActive::clear_remote_reservation(): pending_reservation_nonce 0, reservation_granted false 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] scrubber: FSM: -- state -->> NotActive 2026-02-11T23:31:17.796+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:17.796+0000 7fac51168640 1 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 pct=0'0 crt=107'600 active mbc={}] PeeringState::start_peering_interval up [4,0] -> [4,0], acting [4] -> [4,0], acting_primary 4 -> 4, up_primary 4 -> 4, role -1 -> 1, features acting 4544132024016699391 upacting 4544132024016699391 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] PeeringState::clear_primary_state clear_primary_state 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] clear_primary_state 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] release_backoffs [4:00000000::::0,MAX) 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] scrubber: on_new_interval: current role:Replica/other active?false q/a:false 2026-02-11T23:31:17.796+0000 7fac51168640 20 scrubberFSM event: --vvvv---- IntervalChanged 2026-02-11T23:31:17.796+0000 7fac51168640 20 scrubberFSM event: --^^^^---- IntervalChanged 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] agent_stop 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] on_change 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] cancel_copy_ops 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] cancel_flush_ops 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] cancel_proxy_ops 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] cancel_manifest_ops 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] cancel_cls_gather_ops 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] on_change_cleanup 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] on_change 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] exit NotTrimming 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] enter NotTrimming 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] plpg_on_role_change 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 116 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] handle_activate_map: epoch range: 116..116 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.796+0000 7fac51168640 5 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] exit Reset 0.000091 1 0.000088 2026-02-11T23:31:17.796+0000 7fac51168640 5 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] enter Started 2026-02-11T23:31:17.796+0000 7fac51168640 5 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] enter Start 2026-02-11T23:31:17.796+0000 7fac51168640 1 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] state: transitioning to Stray 2026-02-11T23:31:17.796+0000 7fac51168640 5 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] exit Start 0.000002 0 0.000000 2026-02-11T23:31:17.796+0000 7fac51168640 5 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] enter Started/Stray 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 115 while current is 116 2026-02-11T23:31:17.796+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 116 epoch_requested: 116 MQuery 4.0 from 4 query_epoch 116 query: query(info 0'0 epoch_sent 116) 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::fulfill_info sending info 2026-02-11T23:31:17.796+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 116 get_con_osd_cluster to osd.4 from_epoch 116 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 116 get_nextmap_reserved map_reservations: {116=1} 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 116 release_map epoch: 116 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 116 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 115 -> 116 (shared) 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 116 send_incremental_map epoch range: (116, 116] to v2:127.0.0.1:6836/864727417 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 116 add_map_inc_bl 116 273 bytes 2026-02-11T23:31:17.796+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(116..116 src has 1..116) -- 0x46ea380 con 0x3e4a000 2026-02-11T23:31:17.796+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:116 sent:116 4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) ([106,115] all_participants={0, 4} intervals=[([112,115] acting={4})])) e116/116) -- 0x566aa00 con 0x3e4a000 2026-02-11T23:31:17.796+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:116 sent:116 4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) ([106,115] all_participants={0, 4} intervals=[([112,115] acting={4})])) e116/116) -- 0x566b100 con 0x3e4a000 2026-02-11T23:31:17.796+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.796+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 NullEvt) class_id 2immediate prio 255 cost 10 e116) pg 0x4859000 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.796+0000 7fac55170640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 116 epoch_requested: 116 NullEvt 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 NullEvt) class_id 2immediate prio 255 cost 10 e116) queued 2026-02-11T23:31:17.796+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 116 epoch_requested: 116 NullEvt) class_id 2immediate prio 255 cost 10 e116) pg 0x4a2a000 2026-02-11T23:31:17.796+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] handle_advance_map: 116 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 115 -> 116 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 115 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] state: Started advmap 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 116 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] handle_activate_map: epoch range: 116..116 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 pg_epoch: 116 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 115 while current is 116 2026-02-11T23:31:17.796+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.796+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 116 epoch_requested: 116 NullEvt 2026-02-11T23:31:17.796+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.796+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.796+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.896+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1252 ==== pg_info2(4.0 4.0 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) e117/117) ==== 1085+0+0 (unknown 0 0 0) 0x566b100 con 0x3e4a000 2026-02-11T23:31:17.896+0000 7fac703fa640 15 osd.0 116 enqueue_peering_evt 4.0 epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) pg_lease(ru 0.000000000s ub 76.844123840s int 16.000000000s) 2026-02-11T23:31:17.896+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) pg_lease(ru 0.000000000s ub 76.844123840s int 16.000000000s)) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) pg_lease(ru 0.000000000s ub 76.844123840s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) pg_lease(ru 0.000000000s ub 76.844123840s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) pg 0x4859000 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq _add_slot_waiter 4.0 peering, item epoch is 117, will wait on OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) pg_lease(ru 0.000000000s ub 76.844123840s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.896+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.896+0000 7fac606e4640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1253 ==== osd_map(117..117 src has 1..117) ==== 296+0+0 (unknown 0 0 0) 0x46ea380 con 0x3e4a000 2026-02-11T23:31:17.896+0000 7fac606e4640 20 osd.0 116 OSD::ms_dispatch: osd_map(117..117 src has 1..117) 2026-02-11T23:31:17.896+0000 7fac606e4640 20 osd.0 116 _dispatch 0x46ea380 osd_map(117..117 src has 1..117) 2026-02-11T23:31:17.896+0000 7fac606e4640 3 osd.0 116 handle_osd_map epochs [117,117], i have 116, src has [1,117] 2026-02-11T23:31:17.896+0000 7fac606e4640 10 osd.0 116 handle_osd_map: received 0 full maps and 1 incremental maps 2026-02-11T23:31:17.896+0000 7fac606e4640 10 osd.0 116 handle_osd_map got inc map for epoch 117 2026-02-11T23:31:17.896+0000 7fac606e4640 20 osd.0 116 got_full_map 117, nothing requested 2026-02-11T23:31:17.896+0000 7fac606e4640 20 osd.0 116 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:17.896+0000 7fac606e4640 20 osd.0 116 handle_osd_map pg_num_history pg_num_history(e117 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:31:17.896+0000 7fac606e4640 10 snap_mapper record_purged_snaps purged_snaps {117={}} 2026-02-11T23:31:17.896+0000 7fac606e4640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:17.896+0000 7fac606e4640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e117 maps [1~117] lci=[5,117] tlb=1) 2026-02-11T23:31:17.896+0000 7fac5e983640 10 osd.0 116 _committed_osd_maps 117..117 2026-02-11T23:31:17.896+0000 7fac5e983640 10 osd.0 116 advance to epoch 117 (<= last 117 <= newest_map 117) 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0 117 consume_map version 117 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0 116 await_reserved_maps epoch:117 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0 116 await_reserved_maps done 0 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0 117 consume_map 117 -- publish done 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x4946160 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0 117 identify_splits_and_merges 4.0 e116 to e117 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0 117 identify_splits_and_merges 4.0 e116 to e117 pg_nums {45=1} 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x25809a0 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0 117 identify_splits_and_merges 1.0 e116 to e117 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0 117 identify_splits_and_merges 1.0 e116 to e117 pg_nums {24=1} 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0 117 prune_pg_created 2026-02-11T23:31:17.896+0000 7fac5e983640 10 osd.0:0.consume_map 117 (was 116) 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 pending_peering first epoch 117 <= 117, requeueing 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:0._wake_pg_slot _wake_pg_slot 4.0 to_process <> waiting <> waiting_peering {117=} 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:17.896+0000 7fac5e983640 10 osd.0:1.consume_map 117 (was 116) 2026-02-11T23:31:17.896+0000 7fac5e983640 10 osd.0:2.consume_map 117 (was 116) 2026-02-11T23:31:17.896+0000 7fac5e983640 10 osd.0:3.consume_map 117 (was 116) 2026-02-11T23:31:17.896+0000 7fac5e983640 10 osd.0:4.consume_map 117 (was 116) 2026-02-11T23:31:17.896+0000 7fac5e983640 10 osd.0:5.consume_map 117 (was 116) 2026-02-11T23:31:17.896+0000 7fac5e983640 10 osd.0:6.consume_map 117 (was 116) 2026-02-11T23:31:17.896+0000 7fac5e983640 10 osd.0:7.consume_map 117 (was 116) 2026-02-11T23:31:17.896+0000 7fac5e983640 10 osd.0 117 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:17.896+0000 7fac5e983640 15 osd.0 117 enqueue_peering_evt 4.0 epoch_sent: 117 epoch_requested: 117 NullEvt 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) pg_lease(ru 0.000000000s ub 76.844123840s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 NullEvt) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) pg_lease(ru 0.000000000s ub 76.844123840s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) pg 0x4859000 2026-02-11T23:31:17.896+0000 7fac5e983640 15 osd.0 117 enqueue_peering_evt 1.0 epoch_sent: 117 epoch_requested: 117 NullEvt 2026-02-11T23:31:17.896+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 NullEvt) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] handle_advance_map: 117 2026-02-11T23:31:17.896+0000 7fac5e983640 7 osd.0 117 activate_map version 117 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 116 -> 117 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:31:17.896+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.896+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 NullEvt) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:17.896+0000 7fac55971640 20 osd.0 117 agent_entry empty queue 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] state: Started advmap 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 117 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] handle_activate_map: epoch range: 117..117 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 116 while current is 117 2026-02-11T23:31:17.896+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] do_peering_event: epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) pg_lease(ru 0.000000000s ub 76.844123840s int 16.000000000s) 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] state: got info from osd.4 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::proc_lease pg_lease(ru 0.000000000s ub 76.844123840s int 16.000000000s) 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.887035370s -> ruub 86.731155396s 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] PeeringState::proc_lease readable_until_ub now 86.731155396s 2026-02-11T23:31:17.896+0000 7fac51168640 5 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] exit Started/Stray 0.100357 5 0.000024 2026-02-11T23:31:17.896+0000 7fac51168640 5 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] enter Started/ReplicaActive 2026-02-11T23:31:17.896+0000 7fac51168640 5 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] enter Started/ReplicaActive/RepNotRecovering 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown NOTIFY mbc={}] state: In ReplicaActive, about to call activate 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] PeeringState::activate activate - no missing, moving last_complete 107'600 -> 107'600 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] scrubber: on_replica_activate 2026-02-11T23:31:17.896+0000 7fac51168640 20 scrubberFSM event: --vvvv---- ReplicaActivate 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] scrubber: FSM: -- state -->> ReplicaActive/ReplicaIdle 2026-02-11T23:31:17.896+0000 7fac51168640 20 scrubberFSM event: --^^^^---- ReplicaActivate 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] state: Activate Finished 2026-02-11T23:31:17.896+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 117 get_con_osd_cluster to osd.4 from_epoch 117 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 116 -> 117 (shared) 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 117 send_incremental_map epoch range: (117, 117] to v2:127.0.0.1:6836/864727417 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 117 add_map_inc_bl 117 252 bytes 2026-02-11T23:31:17.896+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(117..117 src has 1..117) -- 0x46ea8c0 con 0x3e4a000 2026-02-11T23:31:17.896+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_notify2(4.0 4.0 (query:117 sent:117 4.0( v 107'600 (107'590,107'600] local-lis/les=112/113 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) ([106,115] all_participants={0, 4} intervals=[([112,115] acting={4})])) e117/117) -- 0x566b800 con 0x3e4a000 2026-02-11T23:31:17.896+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.896+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 NullEvt) class_id 2immediate prio 255 cost 10 e117) pg 0x4859000 2026-02-11T23:31:17.896+0000 7fac55170640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] do_peering_event: epoch_sent: 117 epoch_requested: 117 NullEvt 2026-02-11T23:31:17.896+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 NullEvt) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 NullEvt) class_id 2immediate prio 255 cost 10 e117) pg 0x4a2a000 2026-02-11T23:31:17.896+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] handle_advance_map: 117 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0:0.update_pg_epoch 1.0 116 -> 117 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 116 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] state: Started advmap 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 117 advance_pg new pool opts old pool opts 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] handle_activate_map: epoch range: 117..117 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 pg_epoch: 117 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 115 while current is 117 2026-02-11T23:31:17.896+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.896+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 117 epoch_requested: 117 NullEvt 2026-02-11T23:31:17.896+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.896+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac55170640 15 osd.0 117 enqueue_peering_evt 4.0 epoch_sent: 117 epoch_requested: 117 ActivateCommitted from 117 processed at 117 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 ActivateCommitted from 117 processed at 117) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 ActivateCommitted from 117 processed at 117) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 ActivateCommitted from 117 processed at 117) class_id 2immediate prio 255 cost 10 e117) pg 0x4859000 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac55170640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] do_peering_event: epoch_sent: 117 epoch_requested: 117 ActivateCommitted from 117 processed at 117 2026-02-11T23:31:17.897+0000 7fac55170640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 unknown mbc={}] state: react 117 telling primary 2026-02-11T23:31:17.897+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 117 get_con_osd_cluster to osd.4 from_epoch 117 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:17.897+0000 7fac55170640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:17.897+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_info2(4.0 4.0 4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/106 les/c/f=117/107/0 sis=116) e117/117) -- 0x4b84e00 con 0x3e4a000 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1254 ==== pg_lease(4.0 pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s) e117/117) ==== 58+0+0 (unknown 0 0 0) 0x5664780 con 0x3e4a000 2026-02-11T23:31:17.897+0000 7fac703fa640 15 osd.0 117 enqueue_peering_evt 4.0 epoch_sent: 117 epoch_requested: 117 MLease epoch 117 from osd.4 pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s) 2026-02-11T23:31:17.897+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MLease epoch 117 from osd.4 pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s)) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MLease epoch 117 from osd.4 pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MLease epoch 117 from osd.4 pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) pg 0x4859000 2026-02-11T23:31:17.897+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1255 ==== pg_info2(4.0 4.0 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/106 les/c/f=117/107/0 sis=116) e117/117) ==== 1137+0+0 (unknown 0 0 0) 0x4b84e00 con 0x3e4a000 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac55170640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 117 epoch_requested: 117 MLease epoch 117 from osd.4 pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s) 2026-02-11T23:31:17.897+0000 7fac55170640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s) 2026-02-11T23:31:17.897+0000 7fac703fa640 15 osd.0 117 enqueue_peering_evt 4.0 epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/106 les/c/f=117/107/0 sis=116) pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s) 2026-02-11T23:31:17.897+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/106 les/c/f=117/107/0 sis=116) pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s)) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.886756897s -> ru 86.730873108s 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until now 86.730873108s 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/106 les/c/f=117/107/0 sis=116) pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.887035370s -> ruub 86.732032776s 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 86.732032776s 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 117 get_con_osd_cluster to osd.4 from_epoch 117 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:17.897+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_lease_ack(4.0 pg_lease_ack(ruub 76.844993591s) e117/117) -- 0x56129a0 con 0x3e4a000 2026-02-11T23:31:17.897+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/106 les/c/f=117/107/0 sis=116) pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) pg 0x4859000 2026-02-11T23:31:17.897+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=112/106 les/c/f=113/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/106 les/c/f=117/107/0 sis=116) pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s) 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/106 les/c/f=117/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 active mbc={}] PeeringState::update_history advanced history from ec=45/45 lis/c=116/106 les/c/f=117/107/0 sis=116 2026-02-11T23:31:17.897+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1256 ==== pg_info2(4.0 4.0 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) e117/117) ==== 1137+0+0 (unknown 0 0 0) 0x4b84e00 con 0x3e4a000 2026-02-11T23:31:17.897+0000 7fac703fa640 15 osd.0 117 enqueue_peering_evt 4.0 epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s) 2026-02-11T23:31:17.897+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s)) class_id 2 prio 255 cost 10 e117) 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) queued 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e117) pg 0x4859000 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac55170640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/106 les/c/f=117/107/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 117 epoch_requested: 117 MInfoRec from 4 info: 4.0( v 107'600 (107'500,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) pg_lease(ru 76.844123840s ub 76.844993591s int 16.000000000s) 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 active mbc={}] PeeringState::update_history advanced history from ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) [4,0] r=1 lpr=116 pi=[106,116)/1 crt=107'600 active mbc={}] PeeringState::update_history clearing past_intervals 2026-02-11T23:31:17.897+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:17.897+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:18.013+0000 7fac60ee5640 20 osd.0 117 reports for 0 queries 2026-02-11T23:31:18.013+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x34c3880 con 0x396a800 2026-02-11T23:31:18.013+0000 7fac60ee5640 15 osd.0 117 collect_pg_stats 2026-02-11T23:31:18.013+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836499 v 0) -- 0x3df1200 con 0x396a800 2026-02-11T23:31:18.058+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 20 ==== osd_ping(ping e117 up_from 21 ping_stamp 2026-02-11T23:31:18.059225+0000/61.006732941s send_stamp 61.006732941s delta_ub -9.886732101s) ==== 2033+0+0 (crc 0 0 0) 0x5664780 con 0x3f57400 2026-02-11T23:31:18.058+0000 7fac703fa640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886961937s,-9.886732101s]) 2026-02-11T23:31:18.058+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:18.059225+0000/61.006732941s send_stamp 70.893692017s delta_ub 9.886961937s) -- 0x562b800 con 0x3f57400 2026-02-11T23:31:18.058+0000 7fac703fa640 20 osd.0 117 get_con_osd_cluster to osd.4 from_epoch 117 2026-02-11T23:31:18.058+0000 7fac703fa640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:18.058+0000 7fac703fa640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:18.058+0000 7fac703fa640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:18.058+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 20 ==== osd_ping(ping e117 up_from 21 ping_stamp 2026-02-11T23:31:18.059225+0000/61.006732941s send_stamp 61.006732941s delta_ub -9.886732101s) ==== 2033+0+0 (crc 0 0 0) 0x3e03680 con 0x3f57800 2026-02-11T23:31:18.058+0000 7fac71bfd640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886982918s,-9.886732101s]) 2026-02-11T23:31:18.058+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:18.059225+0000/61.006732941s send_stamp 70.893714905s delta_ub 9.886982918s) -- 0x3e03380 con 0x3f57800 2026-02-11T23:31:18.058+0000 7fac71bfd640 20 osd.0 117 get_con_osd_cluster to osd.4 from_epoch 117 2026-02-11T23:31:18.058+0000 7fac71bfd640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:18.058+0000 7fac71bfd640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:18.058+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:18.105+0000 7fac683ea640 10 osd.0 117 tick_without_osd_lock 2026-02-11T23:31:18.105+0000 7fac683ea640 0 osd.0 117 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:18.105+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:18.105+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:18.105+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:18.105+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.085 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:18.105+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:18.105, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:18.105+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:18.105+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:18.105+0000 7fac683ea640 20 osd.0 117 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:18.105+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:18.105+0000 7fac683ea640 20 osd.0 117 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:18.112+0000 7fac68beb640 10 osd.0 117 tick 2026-02-11T23:31:18.112+0000 7fac68beb640 20 osd.0 117 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:18.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:18.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:18.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:48.171625+0000) 2026-02-11T23:31:19.072+0000 7fac68beb640 10 osd.0 117 tick 2026-02-11T23:31:19.072+0000 7fac68beb640 20 osd.0 117 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:19.084+0000 7fac683ea640 10 osd.0 117 tick_without_osd_lock 2026-02-11T23:31:19.084+0000 7fac683ea640 0 osd.0 117 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:19.084+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:19.084+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.085 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:19.084+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:19.084, recovery is active?:false restrictions:<....> 2026-02-11T23:31:19.084+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:19.084+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:19.084+0000 7fac683ea640 20 osd.0 117 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:19.084+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:19.084+0000 7fac683ea640 20 osd.0 117 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:19.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:19.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:19.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:49.171656+0000) 2026-02-11T23:31:19.251+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 19 ==== osd_ping(ping e117 up_from 9 ping_stamp 2026-02-11T23:31:19.252242+0000/70.105812073s send_stamp 70.105812073s delta_ub -1.980729818s) ==== 2033+0+0 (crc 0 0 0) 0x3e03380 con 0x396a400 2026-02-11T23:31:19.251+0000 7fac71bfd640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980919242s,-1.980729818s]) 2026-02-11T23:31:19.251+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:19.252242+0000/70.105812073s send_stamp 72.086723328s delta_ub 1.980919242s) -- 0x3e03680 con 0x396a400 2026-02-11T23:31:19.251+0000 7fac71bfd640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:19.251+0000 7fac71bfd640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:19.251+0000 7fac71bfd640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:19.251+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 115 to ping map epoch of 117 2026-02-11T23:31:19.251+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:19.251+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 19 ==== osd_ping(ping e117 up_from 9 ping_stamp 2026-02-11T23:31:19.252242+0000/70.105812073s send_stamp 70.105812073s delta_ub -1.980729818s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x396a000 2026-02-11T23:31:19.251+0000 7fac723fe640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980957270s,-1.980729818s]) 2026-02-11T23:31:19.251+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:19.252242+0000/70.105812073s send_stamp 72.086769104s delta_ub 1.980957270s) -- 0x3e03980 con 0x396a000 2026-02-11T23:31:19.251+0000 7fac723fe640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:19.251+0000 7fac723fe640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:19.251+0000 7fac723fe640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:19.251+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:20.111+0000 7fac68beb640 10 osd.0 117 tick 2026-02-11T23:31:20.111+0000 7fac68beb640 20 osd.0 117 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:20.126+0000 7fac683ea640 10 osd.0 117 tick_without_osd_lock 2026-02-11T23:31:20.126+0000 7fac683ea640 0 osd.0 117 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:20.126+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:20.126+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:20.126+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:20.126+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.078 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:20.126+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:20.126, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:20.126+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:20.126+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:20.126+0000 7fac683ea640 20 osd.0 117 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:20.126+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:20.126+0000 7fac683ea640 20 osd.0 117 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:20.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:20.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:20.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:50.171682+0000) 2026-02-11T23:31:20.171+0000 7fac616e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:20.282+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 20 ==== osd_ping(ping e117 up_from 17 ping_stamp 2026-02-11T23:31:20.282801+0000/65.806838989s send_stamp 65.806838989s delta_ub -7.310208321s) ==== 2033+0+0 (crc 0 0 0) 0x3e03680 con 0x3f56800 2026-02-11T23:31:20.282+0000 7fac71bfd640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310436726s,-7.310208321s]) 2026-02-11T23:31:20.282+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:20.282801+0000/65.806838989s send_stamp 73.117279053s delta_ub 7.310436726s) -- 0x3e03380 con 0x3f56800 2026-02-11T23:31:20.282+0000 7fac71bfd640 20 osd.0 117 get_con_osd_cluster to osd.3 from_epoch 117 2026-02-11T23:31:20.282+0000 7fac71bfd640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:20.282+0000 7fac71bfd640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:20.282+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 107 to ping map epoch of 117 2026-02-11T23:31:20.282+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:20.282+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 20 ==== osd_ping(ping e117 up_from 17 ping_stamp 2026-02-11T23:31:20.282801+0000/65.806838989s send_stamp 65.806838989s delta_ub -7.310208321s) ==== 2033+0+0 (crc 0 0 0) 0x3e03980 con 0x3f56c00 2026-02-11T23:31:20.282+0000 7fac723fe640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310461521s,-7.310208321s]) 2026-02-11T23:31:20.282+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:20.282801+0000/65.806838989s send_stamp 73.117301941s delta_ub 7.310461521s) -- 0x3e02480 con 0x3f56c00 2026-02-11T23:31:20.282+0000 7fac723fe640 20 osd.0 117 get_con_osd_cluster to osd.3 from_epoch 117 2026-02-11T23:31:20.282+0000 7fac723fe640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:20.282+0000 7fac723fe640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:20.282+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:21.158+0000 7fac68beb640 10 osd.0 117 tick 2026-02-11T23:31:21.158+0000 7fac68beb640 20 osd.0 117 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:21.168+0000 7fac683ea640 10 osd.0 117 tick_without_osd_lock 2026-02-11T23:31:21.168+0000 7fac683ea640 0 osd.0 117 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:21.168+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:21.168+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:21.168+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:21.168+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.078 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:21.168+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:21.169, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:21.168+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:21.168+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:21.168+0000 7fac683ea640 20 osd.0 117 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:21.168+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:21.168+0000 7fac683ea640 20 osd.0 117 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:21.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:21.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:21.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:51.171726+0000) 2026-02-11T23:31:21.382+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 21 ==== osd_ping(ping e117 up_from 17 ping_stamp 2026-02-11T23:31:21.382972+0000/66.907005310s send_stamp 66.907005310s delta_ub -7.310316086s) ==== 2033+0+0 (crc 0 0 0) 0x3e03380 con 0x3f56800 2026-02-11T23:31:21.382+0000 7fac71bfd640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310430527s,-7.310316086s]) 2026-02-11T23:31:21.382+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:21.382972+0000/66.907005310s send_stamp 74.217437744s delta_ub 7.310430527s) -- 0x3e03680 con 0x3f56800 2026-02-11T23:31:21.382+0000 7fac71bfd640 20 osd.0 117 get_con_osd_cluster to osd.3 from_epoch 117 2026-02-11T23:31:21.382+0000 7fac71bfd640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:21.382+0000 7fac71bfd640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:21.382+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:21.382+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 21 ==== osd_ping(ping e117 up_from 17 ping_stamp 2026-02-11T23:31:21.382972+0000/66.907005310s send_stamp 66.907005310s delta_ub -7.310316086s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x3f56c00 2026-02-11T23:31:21.382+0000 7fac723fe640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310448170s,-7.310316086s]) 2026-02-11T23:31:21.382+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:21.382972+0000/66.907005310s send_stamp 74.217453003s delta_ub 7.310448170s) -- 0x3e03980 con 0x3f56c00 2026-02-11T23:31:21.382+0000 7fac723fe640 20 osd.0 117 get_con_osd_cluster to osd.3 from_epoch 117 2026-02-11T23:31:21.382+0000 7fac723fe640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:21.382+0000 7fac723fe640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:21.382+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:21.552+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 20 ==== osd_ping(ping e117 up_from 9 ping_stamp 2026-02-11T23:31:21.552470+0000/72.406028748s send_stamp 72.406028748s delta_ub -1.980772734s) ==== 2033+0+0 (crc 0 0 0) 0x3e03680 con 0x396a400 2026-02-11T23:31:21.552+0000 7fac71bfd640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980911493s,-1.980772734s]) 2026-02-11T23:31:21.552+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 20 ==== osd_ping(ping e117 up_from 9 ping_stamp 2026-02-11T23:31:21.552470+0000/72.406028748s send_stamp 72.406028748s delta_ub -1.980772734s) ==== 2033+0+0 (crc 0 0 0) 0x3e03980 con 0x396a000 2026-02-11T23:31:21.552+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:21.552470+0000/72.406028748s send_stamp 74.386947632s delta_ub 1.980911493s) -- 0x3e03380 con 0x396a400 2026-02-11T23:31:21.552+0000 7fac71bfd640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:21.552+0000 7fac71bfd640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:21.552+0000 7fac71bfd640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:21.552+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:21.552+0000 7fac723fe640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980927944s,-1.980772734s]) 2026-02-11T23:31:21.552+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:21.552470+0000/72.406028748s send_stamp 74.386962891s delta_ub 1.980927944s) -- 0x3e02480 con 0x396a000 2026-02-11T23:31:21.552+0000 7fac723fe640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:21.552+0000 7fac723fe640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:21.552+0000 7fac723fe640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:21.552+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.020+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 18 ==== osd_ping(ping e117 up_from 13 ping_stamp 2026-02-11T23:31:22.020638+0000/70.206031799s send_stamp 70.206031799s delta_ub -4.648870945s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x3f56000 2026-02-11T23:31:22.020+0000 7fac723fe640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649092674s,-4.648870945s]) 2026-02-11T23:31:22.020+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:22.020638+0000/70.206031799s send_stamp 74.855125427s delta_ub 4.649092674s) -- 0x3e03980 con 0x3f56000 2026-02-11T23:31:22.020+0000 7fac723fe640 20 osd.0 117 get_con_osd_cluster to osd.2 from_epoch 117 2026-02-11T23:31:22.020+0000 7fac723fe640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.020+0000 7fac723fe640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.020+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 107 to ping map epoch of 117 2026-02-11T23:31:22.020+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.020+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 18 ==== osd_ping(ping e117 up_from 13 ping_stamp 2026-02-11T23:31:22.020638+0000/70.206031799s send_stamp 70.206031799s delta_ub -4.648870945s) ==== 2033+0+0 (crc 0 0 0) 0x562b800 con 0x3f56400 2026-02-11T23:31:22.020+0000 7fac703fa640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649109840s,-4.648870945s]) 2026-02-11T23:31:22.020+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:22.020638+0000/70.206031799s send_stamp 74.855148315s delta_ub 4.649109840s) -- 0x5664780 con 0x3f56400 2026-02-11T23:31:22.020+0000 7fac703fa640 20 osd.0 117 get_con_osd_cluster to osd.2 from_epoch 117 2026-02-11T23:31:22.020+0000 7fac703fa640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.020+0000 7fac703fa640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.020+0000 7fac703fa640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.052+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 21 ==== osd_ping(ping e117 up_from 9 ping_stamp 2026-02-11T23:31:22.052646+0000/72.906211853s send_stamp 72.906211853s delta_ub -1.980811596s) ==== 2033+0+0 (crc 0 0 0) 0x3e03380 con 0x396a400 2026-02-11T23:31:22.052+0000 7fac71bfd640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980890393s,-1.980811596s]) 2026-02-11T23:31:22.052+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 21 ==== osd_ping(ping e117 up_from 9 ping_stamp 2026-02-11T23:31:22.052646+0000/72.906211853s send_stamp 72.906211853s delta_ub -1.980811596s) ==== 2033+0+0 (crc 0 0 0) 0x3e03980 con 0x396a000 2026-02-11T23:31:22.052+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:22.052646+0000/72.906211853s send_stamp 74.887100220s delta_ub 1.980890393s) -- 0x3e03680 con 0x396a400 2026-02-11T23:31:22.052+0000 7fac71bfd640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:22.052+0000 7fac71bfd640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.052+0000 7fac71bfd640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.052+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.052+0000 7fac723fe640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980904937s,-1.980811596s]) 2026-02-11T23:31:22.052+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:22.052646+0000/72.906211853s send_stamp 74.887115479s delta_ub 1.980904937s) -- 0x3e02480 con 0x396a000 2026-02-11T23:31:22.052+0000 7fac723fe640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:22.052+0000 7fac723fe640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.052+0000 7fac723fe640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.052+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.109+0000 7fac68beb640 10 osd.0 117 tick 2026-02-11T23:31:22.109+0000 7fac68beb640 20 osd.0 117 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:22.120+0000 7fac683ea640 10 osd.0 117 tick_without_osd_lock 2026-02-11T23:31:22.120+0000 7fac683ea640 0 osd.0 117 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:22.120+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:22.120+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.078 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:22.120+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:22.121, recovery is active?:false restrictions:<....> 2026-02-11T23:31:22.120+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:22.120+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:22.120+0000 7fac683ea640 20 osd.0 117 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:22.120+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:22.120+0000 7fac683ea640 20 osd.0 117 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:22.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:22.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:22.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:52.171753+0000) 2026-02-11T23:31:22.672+0000 7fac4d160640 5 osd.0 117 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x536a06/0x5dc000, compress 0x0/0x0/0x0, omap 0xbffc, meta 0x1a44004), peers [1,2,3,4] op hist []) 2026-02-11T23:31:22.672+0000 7fac4d160640 20 osd.0 117 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:22.672+0000 7fac4d160640 10 osd.0 117 check_full_status none -> failsafe 2026-02-11T23:31:22.672+0000 7fac4d160640 -1 log_channel(cluster) log [ERR] : full status failsafe engaged, dropping updates, now 100% full 2026-02-11T23:31:22.672+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e117 up_from 5 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 75.507591248s delta_ub 1.980904937s) -- 0x3da2a80 con 0x396b400 2026-02-11T23:31:22.672+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e117 up_from 5 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 75.507591248s delta_ub 1.980904937s) -- 0x3da2300 con 0x3752400 2026-02-11T23:31:22.672+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e117 up_from 5 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 75.507591248s delta_ub 4.649109840s) -- 0x3da3680 con 0x396b800 2026-02-11T23:31:22.672+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e117 up_from 5 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 75.507591248s delta_ub 4.649109840s) -- 0x3da3380 con 0x34c1000 2026-02-11T23:31:22.672+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e117 up_from 5 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 75.507591248s delta_ub 7.310448170s) -- 0x3b1bb00 con 0x3740000 2026-02-11T23:31:22.672+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e117 up_from 5 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 75.507591248s delta_ub 7.310448170s) -- 0x3b1af00 con 0x3740400 2026-02-11T23:31:22.672+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e117 up_from 5 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 75.507591248s delta_ub 9.886982918s) -- 0x3b1a000 con 0x3740800 2026-02-11T23:31:22.672+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e117 up_from 5 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 75.507591248s delta_ub 9.886982918s) -- 0x3b1b380 con 0x3740c00 2026-02-11T23:31:22.672+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 24 ==== osd_ping(ping_reply e117 up_from 9 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 73.526794434s delta_ub -1.980788827s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x396b400 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.672+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 24 ==== osd_ping(ping_reply e117 up_from 9 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 73.526809692s delta_ub -1.980770946s) ==== 2033+0+0 (crc 0 0 0) 0x3e02180 con 0x3752400 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.672+0000 7fac703fa640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980914950s,-1.980788827s]) 2026-02-11T23:31:22.672+0000 7fac71bfd640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:22.672+0000 7fac71bfd640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.672+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 23 ==== osd_ping(ping_reply e117 up_from 13 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 70.858650208s delta_ub -4.648930073s) ==== 2033+0+0 (crc 0 0 0) 0x3da2900 con 0x396b800 2026-02-11T23:31:22.672+0000 7fac71bfd640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.672+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.672+0000 7fac71bfd640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980907440s,-1.980770946s]) 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 get_con_osd_cluster to osd.2 from_epoch 117 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.672+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.672+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 21 ==== osd_ping(ping_reply e117 up_from 21 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 65.620788574s delta_ub -9.886793137s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x3740c00 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649080753s,-4.648930073s]) 2026-02-11T23:31:22.672+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 23 ==== osd_ping(ping_reply e117 up_from 13 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 70.858680725s delta_ub -4.648911476s) ==== 2033+0+0 (crc 0 0 0) 0x3e02180 con 0x34c1000 2026-02-11T23:31:22.672+0000 7fac71bfd640 20 osd.0 117 get_con_osd_cluster to osd.2 from_epoch 117 2026-02-11T23:31:22.672+0000 7fac71bfd640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.672+0000 7fac71bfd640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.672+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.672+0000 7fac71bfd640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649069786s,-4.648911476s]) 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 get_con_osd_cluster to osd.4 from_epoch 117 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.672+0000 7fac703fa640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886958122s,-9.886793137s]) 2026-02-11T23:31:22.672+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 21 ==== osd_ping(ping_reply e117 up_from 17 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 68.197311401s delta_ub -7.310276031s) ==== 2033+0+0 (crc 0 0 0) 0x3da2900 con 0x3740400 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 get_con_osd_cluster to osd.3 from_epoch 117 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.672+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310447216s,-7.310276031s]) 2026-02-11T23:31:22.672+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 21 ==== osd_ping(ping_reply e117 up_from 21 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 65.620811462s delta_ub -9.886775017s) ==== 2033+0+0 (crc 0 0 0) 0x3da2900 con 0x3740800 2026-02-11T23:31:22.672+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 21 ==== osd_ping(ping_reply e117 up_from 17 ping_stamp 2026-02-11T23:31:22.673170+0000/75.507591248s send_stamp 68.197326660s delta_ub -7.310260773s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x3740000 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 get_con_osd_cluster to osd.4 from_epoch 117 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.672+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.672+0000 7fac723fe640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886958122s,-9.886775017s]) 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 get_con_osd_cluster to osd.3 from_epoch 117 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.672+0000 7fac703fa640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.672+0000 7fac703fa640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310456753s,-7.310260773s]) 2026-02-11T23:31:22.758+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 21 ==== osd_ping(ping e117 up_from 21 ping_stamp 2026-02-11T23:31:22.759354+0000/65.706863403s send_stamp 65.706863403s delta_ub -9.886775017s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x3f57400 2026-02-11T23:31:22.758+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 21 ==== osd_ping(ping e117 up_from 21 ping_stamp 2026-02-11T23:31:22.759354+0000/65.706863403s send_stamp 65.706863403s delta_ub -9.886775017s) ==== 2033+0+0 (crc 0 0 0) 0x3e02180 con 0x3f57800 2026-02-11T23:31:22.758+0000 7fac703fa640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886955261s,-9.886775017s]) 2026-02-11T23:31:22.759+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:22.759354+0000/65.706863403s send_stamp 75.593818665s delta_ub 9.886955261s) -- 0x3b1bb00 con 0x3f57400 2026-02-11T23:31:22.759+0000 7fac703fa640 20 osd.0 117 get_con_osd_cluster to osd.4 from_epoch 117 2026-02-11T23:31:22.759+0000 7fac703fa640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.759+0000 7fac703fa640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.759+0000 7fac703fa640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:22.759+0000 7fac71bfd640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886977196s,-9.886775017s]) 2026-02-11T23:31:22.759+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:22.759354+0000/65.706863403s send_stamp 75.593833923s delta_ub 9.886977196s) -- 0x3e02300 con 0x3f57800 2026-02-11T23:31:22.759+0000 7fac71bfd640 20 osd.0 117 get_con_osd_cluster to osd.4 from_epoch 117 2026-02-11T23:31:22.759+0000 7fac71bfd640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:22.759+0000 7fac71bfd640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:22.759+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:23.014+0000 7fac60ee5640 20 osd.0 117 reports for 0 queries 2026-02-11T23:31:23.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x389ce00 con 0x396a800 2026-02-11T23:31:23.014+0000 7fac60ee5640 15 osd.0 117 collect_pg_stats 2026-02-11T23:31:23.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836500 v 0) -- 0x3df1500 con 0x396a800 2026-02-11T23:31:23.082+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 22 ==== osd_ping(ping e117 up_from 17 ping_stamp 2026-02-11T23:31:23.083140+0000/68.607177734s send_stamp 68.607177734s delta_ub -7.310260773s) ==== 2033+0+0 (crc 0 0 0) 0x3e02300 con 0x3f56800 2026-02-11T23:31:23.082+0000 7fac71bfd640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310421944s,-7.310260773s]) 2026-02-11T23:31:23.082+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:23.083140+0000/68.607177734s send_stamp 75.917594910s delta_ub 7.310421944s) -- 0x3e02180 con 0x3f56800 2026-02-11T23:31:23.082+0000 7fac71bfd640 20 osd.0 117 get_con_osd_cluster to osd.3 from_epoch 117 2026-02-11T23:31:23.082+0000 7fac71bfd640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:23.082+0000 7fac71bfd640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:23.082+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:23.082+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 22 ==== osd_ping(ping e117 up_from 17 ping_stamp 2026-02-11T23:31:23.083140+0000/68.607177734s send_stamp 68.607177734s delta_ub -7.310260773s) ==== 2033+0+0 (crc 0 0 0) 0x3da2900 con 0x3f56c00 2026-02-11T23:31:23.082+0000 7fac723fe640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310435772s,-7.310260773s]) 2026-02-11T23:31:23.082+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:23.083140+0000/68.607177734s send_stamp 75.917617798s delta_ub 7.310435772s) -- 0x3da2480 con 0x3f56c00 2026-02-11T23:31:23.082+0000 7fac723fe640 20 osd.0 117 get_con_osd_cluster to osd.3 from_epoch 117 2026-02-11T23:31:23.082+0000 7fac723fe640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:23.082+0000 7fac723fe640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:23.082+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:23.105+0000 7fac68beb640 10 osd.0 117 tick 2026-02-11T23:31:23.105+0000 7fac68beb640 20 osd.0 117 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:23.152+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 22 ==== osd_ping(ping e117 up_from 9 ping_stamp 2026-02-11T23:31:23.152772+0000/74.006340027s send_stamp 74.006340027s delta_ub -1.980770946s) ==== 2033+0+0 (crc 0 0 0) 0x3e02180 con 0x396a400 2026-02-11T23:31:23.152+0000 7fac71bfd640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980893970s,-1.980770946s]) 2026-02-11T23:31:23.152+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 22 ==== osd_ping(ping e117 up_from 9 ping_stamp 2026-02-11T23:31:23.152772+0000/74.006340027s send_stamp 74.006340027s delta_ub -1.980770946s) ==== 2033+0+0 (crc 0 0 0) 0x3da2480 con 0x396a000 2026-02-11T23:31:23.152+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:23.152772+0000/74.006340027s send_stamp 75.987228394s delta_ub 1.980893970s) -- 0x3e02300 con 0x396a400 2026-02-11T23:31:23.152+0000 7fac71bfd640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:23.152+0000 7fac71bfd640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:23.152+0000 7fac71bfd640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:23.152+0000 7fac71bfd640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:23.152+0000 7fac723fe640 20 osd.0 117 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980907440s,-1.980770946s]) 2026-02-11T23:31:23.152+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e117 up_from 5 ping_stamp 2026-02-11T23:31:23.152772+0000/74.006340027s send_stamp 75.987243652s delta_ub 1.980907440s) -- 0x3da2900 con 0x396a000 2026-02-11T23:31:23.152+0000 7fac723fe640 20 osd.0 117 get_con_osd_cluster to osd.1 from_epoch 117 2026-02-11T23:31:23.152+0000 7fac723fe640 20 osd.0 117 get_nextmap_reserved map_reservations: {117=1} 2026-02-11T23:31:23.152+0000 7fac723fe640 20 osd.0 117 release_map epoch: 117 2026-02-11T23:31:23.152+0000 7fac723fe640 10 osd.0 117 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 117 is not newer than session's projected_epoch of 117 2026-02-11T23:31:23.154+0000 7fac683ea640 10 osd.0 117 tick_without_osd_lock 2026-02-11T23:31:23.154+0000 7fac683ea640 0 osd.0 117 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:23.154+0000 7fac683ea640 10 osd.0 117 send_full_update want state full 2026-02-11T23:31:23.154+0000 7fac683ea640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:23.154+0000 7fac683ea640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- osd_full(e117 full v117) -- 0x5613340 con 0x396bc00 2026-02-11T23:31:23.154+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:23.154+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:23.154+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:23.154+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.078 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:23.154+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:23.155, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:23.154+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:23.154+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:23.154+0000 7fac683ea640 20 osd.0 117 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:23.154+0000 7fac683ea640 10 osd.0 117 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:23.154+0000 7fac683ea640 20 osd.0 117 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:23.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:23.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:23.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:53.171779+0000) 2026-02-11T23:31:23.171+0000 7fac616e6640 10 log_client log_queue is 1 last_log 3 sent 2 num 1 unsent 1 sending 1 2026-02-11T23:31:23.171+0000 7fac616e6640 10 log_client will send 2026-02-11T23:31:22.673167+0000 osd.0 (osd.0) 3 : cluster [ERR] full status failsafe engaged, dropping updates, now 100% full 2026-02-11T23:31:23.171+0000 7fac616e6640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:23.171+0000 7fac616e6640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- log(1 entries from seq 3 at 2026-02-11T23:31:22.673167+0000) -- 0x5678000 con 0x396bc00 2026-02-11T23:31:23.205+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 58 ==== osd_map(117..118 src has 1..118) ==== 1216+0+0 (crc 0 0 0) 0x5678000 con 0x396bc00 2026-02-11T23:31:23.205+0000 7fac626e8640 20 osd.0 117 OSD::ms_dispatch: osd_map(117..118 src has 1..118) 2026-02-11T23:31:23.205+0000 7fac626e8640 20 osd.0 117 _dispatch 0x5678000 osd_map(117..118 src has 1..118) 2026-02-11T23:31:23.205+0000 7fac626e8640 10 monclient: _renew_subs 2026-02-11T23:31:23.205+0000 7fac626e8640 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:7180/0 2026-02-11T23:31:23.205+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> v2:127.0.0.1:7180/0 -- mon_subscribe({osdmap=119}) -- 0x449dba0 con 0x396bc00 2026-02-11T23:31:23.205+0000 7fac626e8640 3 osd.0 117 handle_osd_map epochs [117,118], i have 117, src has [1,118] 2026-02-11T23:31:23.205+0000 7fac626e8640 10 osd.0 117 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:31:23.205+0000 7fac626e8640 10 osd.0 117 handle_osd_map got inc map for epoch 118 2026-02-11T23:31:23.205+0000 7fac626e8640 20 osd.0 117 got_full_map 118, nothing requested 2026-02-11T23:31:23.205+0000 7fac626e8640 20 osd.0 117 trim_maps: min=1 oldest_map=1 2026-02-11T23:31:23.205+0000 7fac626e8640 20 osd.0 117 handle_osd_map pg_num_history pg_num_history(e118 pg_nums {1={24=1},2={31=1},3={38=1},4={45=1},5={52=1},6={59=1},7={66=1},8={72=1},9={81=1},10={88=1}} deleted_pools 95,2,96,3,97,5,98,6,99,7,100,8,101,9,102,10) 2026-02-11T23:31:23.205+0000 7fac626e8640 10 snap_mapper record_purged_snaps purged_snaps {118={}} 2026-02-11T23:31:23.205+0000 7fac626e8640 10 snap_mapper record_purged_snaps rm 0 keys, set 0 keys 2026-02-11T23:31:23.205+0000 7fac626e8640 10 write_superblock sb(c7e182d9-676d-4eeb-80b1-915a88d421e5 osd.0 37ab9e04-8330-421e-b80d-6f0af31f81ef e118 maps [1~118] lci=[5,118] tlb=1) 2026-02-11T23:31:23.206+0000 7fac5e983640 10 osd.0 117 _committed_osd_maps 118..118 2026-02-11T23:31:23.206+0000 7fac5e983640 10 osd.0 117 advance to epoch 118 (<= last 118 <= newest_map 118) 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0 118 consume_map version 118 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0 117 await_reserved_maps epoch:118 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0 117 await_reserved_maps done 0 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0 118 consume_map 118 -- publish done 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 2 slots 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:4.0slot:0x4946160 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0 118 identify_splits_and_merges 4.0 e117 to e118 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0 118 identify_splits_and_merges 4.0 e117 to e118 pg_nums {45=1} 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges slot pgid:1.0slot:0x25809a0 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0 118 identify_splits_and_merges 1.0 e117 to e118 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0 118 identify_splits_and_merges 1.0 e117 to e118 pg_nums {24=1} 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:0.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:1.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:2.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:3.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:4.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:5.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:6.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 slots 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:7.identify_splits_and_merges identify_splits_and_merges 0 splits, 0 merges 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0 118 prune_pg_created 2026-02-11T23:31:23.206+0000 7fac5e983640 10 osd.0:0.consume_map 118 (was 117) 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 4.0 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0:0.consume_map consume_map 1.0 2026-02-11T23:31:23.206+0000 7fac5e983640 10 osd.0:1.consume_map 118 (was 117) 2026-02-11T23:31:23.206+0000 7fac5e983640 10 osd.0:2.consume_map 118 (was 117) 2026-02-11T23:31:23.206+0000 7fac5e983640 10 osd.0:3.consume_map 118 (was 117) 2026-02-11T23:31:23.206+0000 7fac5e983640 10 osd.0:4.consume_map 118 (was 117) 2026-02-11T23:31:23.206+0000 7fac5e983640 10 osd.0:5.consume_map 118 (was 117) 2026-02-11T23:31:23.206+0000 7fac5e983640 10 osd.0:6.consume_map 118 (was 117) 2026-02-11T23:31:23.206+0000 7fac5e983640 10 osd.0:7.consume_map 118 (was 117) 2026-02-11T23:31:23.206+0000 7fac5e983640 10 osd.0 118 release_reserved_pushes(0), recovery_ops_reserved 0 -> 0 2026-02-11T23:31:23.206+0000 7fac5e983640 15 osd.0 118 enqueue_peering_evt 4.0 epoch_sent: 118 epoch_requested: 118 NullEvt 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 NullEvt) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:23.206+0000 7fac5e983640 15 osd.0 118 enqueue_peering_evt 1.0 epoch_sent: 118 epoch_requested: 118 NullEvt 2026-02-11T23:31:23.206+0000 7fac5e983640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 NullEvt) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:23.206+0000 7fac5e983640 7 osd.0 118 activate_map version 118 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 NullEvt) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:23.206+0000 7fac55971640 20 osd.0 118 agent_entry empty queue 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 NullEvt) class_id 2immediate prio 255 cost 10 e118) pg 0x4859000 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 NullEvt) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 NullEvt) class_id 2immediate prio 255 cost 10 e118) pg 0x4a2a000 2026-02-11T23:31:23.206+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] handle_advance_map: 118 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0:0.update_pg_epoch 4.0 117 -> 118 2026-02-11T23:31:23.206+0000 7fac51168640 10 osd.0 pg_epoch: 117 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::advance_map handle_advance_map [4,0]/[4,0] -- 4/4 2026-02-11T23:31:23.206+0000 7fac55170640 10 osd.0 pg_epoch: 117 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] handle_advance_map: 118 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0:0.update_pg_epoch 1.0 117 -> 118 2026-02-11T23:31:23.206+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] state: Started advmap 2026-02-11T23:31:23.206+0000 7fac55170640 10 osd.0 pg_epoch: 117 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::advance_map handle_advance_map [1,0]/[1,0] -- 1/1 2026-02-11T23:31:23.206+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/0 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::check_full_transition pool was marked full in 118 2026-02-11T23:31:23.206+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:23.206+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] state: Started advmap 2026-02-11T23:31:23.206+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/0 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::check_full_transition pool was marked full in 118 2026-02-11T23:31:23.206+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] plpg_on_pool_change 2026-02-11T23:31:23.206+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] check_recovery_sources no source osds () went down 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] hit_set_clear 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] agent_stop 2026-02-11T23:31:23.206+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] plpg_on_pool_change 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0 118 advance_pg new pool opts old pool opts 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] hit_set_clear 2026-02-11T23:31:23.206+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] handle_activate_map: epoch range: 118..118 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] agent_stop 2026-02-11T23:31:23.206+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0 118 advance_pg new pool opts old pool opts 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 117 while current is 118 2026-02-11T23:31:23.206+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] handle_activate_map: epoch range: 118..118 2026-02-11T23:31:23.206+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::activate_map 2026-02-11T23:31:23.206+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::activate_map : Not dirtying info: last_persisted is 115 while current is 118 2026-02-11T23:31:23.206+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:31:23.206+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 NullEvt 2026-02-11T23:31:23.206+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:23.206+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:23.206+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] on_scrub_schedule_input_change: inactive, non-primary - or already scrubbing 2026-02-11T23:31:23.206+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 NullEvt 2026-02-11T23:31:23.206+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:23.206+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:23.206+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:23.206+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 59 ==== osd_map(117..118 src has 1..118) ==== 1216+0+0 (crc 0 0 0) 0x46ebc00 con 0x396bc00 2026-02-11T23:31:23.206+0000 7fac626e8640 20 osd.0 118 OSD::ms_dispatch: osd_map(117..118 src has 1..118) 2026-02-11T23:31:23.206+0000 7fac626e8640 20 osd.0 118 _dispatch 0x46ebc00 osd_map(117..118 src has 1..118) 2026-02-11T23:31:23.206+0000 7fac626e8640 3 osd.0 118 handle_osd_map epochs [117,118], i have 118, src has [1,118] 2026-02-11T23:31:23.206+0000 7fac626e8640 10 osd.0 118 handle_osd_map: received 0 full maps and 2 incremental maps 2026-02-11T23:31:23.206+0000 7fac626e8640 10 osd.0 118 no new maps here, dropping 2026-02-11T23:31:23.221+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== mon.0 v2:127.0.0.1:7180/0 60 ==== log(last 3) ==== 28+0+0 (crc 0 0 0) 0x3da2900 con 0x396bc00 2026-02-11T23:31:23.221+0000 7fac626e8640 10 log_client handle_log_ack log(last 3) 2026-02-11T23:31:23.221+0000 7fac626e8640 10 log_client logged 2026-02-11T23:31:22.673167+0000 osd.0 (osd.0) 3 : cluster [ERR] full status failsafe engaged, dropping updates, now 100% full 2026-02-11T23:31:23.859+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 22 ==== osd_ping(ping e117 up_from 21 ping_stamp 2026-02-11T23:31:23.859587+0000/66.807098389s send_stamp 66.807098389s delta_ub -9.886796951s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bb00 con 0x3f57400 2026-02-11T23:31:23.859+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886962891s,-9.886796951s]) 2026-02-11T23:31:23.859+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:23.859587+0000/66.807098389s send_stamp 76.694053650s delta_ub 9.886962891s) -- 0x3b1b380 con 0x3f57400 2026-02-11T23:31:23.859+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:23.859+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:23.859+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:23.859+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 map epoch 117 -> 118 (shared) 2026-02-11T23:31:23.859+0000 7fac703fa640 10 osd.0 118 send_incremental_map epoch range: (118, 118] to v2:127.0.0.1:6836/864727417 2026-02-11T23:31:23.859+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 22 ==== osd_ping(ping e117 up_from 21 ping_stamp 2026-02-11T23:31:23.859587+0000/66.807098389s send_stamp 66.807098389s delta_ub -9.886796951s) ==== 2033+0+0 (crc 0 0 0) 0x3e02300 con 0x3f57800 2026-02-11T23:31:23.859+0000 7fac703fa640 10 osd.0 118 add_map_inc_bl 118 912 bytes 2026-02-11T23:31:23.859+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- osd_map(118..118 src has 1..118) -- 0x46ea8c0 con 0x3e4a000 2026-02-11T23:31:23.859+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887121201s,-9.886796951s]) 2026-02-11T23:31:23.859+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:23.859587+0000/66.807098389s send_stamp 76.694213867s delta_ub 9.887121201s) -- 0x3e02180 con 0x3f57800 2026-02-11T23:31:23.859+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:23.859+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:23.859+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:23.859+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:24.122+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:24.122+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:24.122+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:24.122+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:24.122+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:24.122+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.078 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:24.122+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:24.122, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:24.122+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:24.122+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:24.122+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:24.122+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:24.122+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:24.130+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:24.130+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:24.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:24.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:24.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:54.171841+0000) 2026-02-11T23:31:24.182+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 23 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:24.183273+0000/69.707305908s send_stamp 69.707305908s delta_ub -7.310330391s) ==== 2033+0+0 (crc 0 0 0) 0x3e02180 con 0x3f56800 2026-02-11T23:31:24.182+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 23 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:24.183273+0000/69.707305908s send_stamp 69.707305908s delta_ub -7.310330391s) ==== 2033+0+0 (crc 0 0 0) 0x3da2480 con 0x3f56c00 2026-02-11T23:31:24.182+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310436249s,-7.310330391s]) 2026-02-11T23:31:24.182+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:24.183273+0000/69.707305908s send_stamp 77.017745972s delta_ub 7.310436249s) -- 0x3e02300 con 0x3f56800 2026-02-11T23:31:24.182+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:24.182+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:24.182+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:24.182+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 updating session's projected_epoch from 117 to ping map epoch of 118 2026-02-11T23:31:24.182+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:24.182+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310456276s,-7.310330391s]) 2026-02-11T23:31:24.182+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:24.183273+0000/69.707305908s send_stamp 77.017768860s delta_ub 7.310456276s) -- 0x3da3680 con 0x3f56c00 2026-02-11T23:31:24.182+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:24.182+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:24.182+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:24.182+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:25.083+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:25.083+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:25.170+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:25.170+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:25.170+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:25.170+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:25.170+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:25.170+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.072 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:25.170+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:25.170, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:25.170+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:25.170+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:25.170+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:25.170+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:25.170+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:25.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:25.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:25.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:55.171865+0000) 2026-02-11T23:31:25.452+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 23 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:25.452906+0000/76.306472778s send_stamp 76.306472778s delta_ub -1.980817080s) ==== 2033+0+0 (crc 0 0 0) 0x3e02300 con 0x396a400 2026-02-11T23:31:25.452+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980909944s,-1.980817080s]) 2026-02-11T23:31:25.452+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:25.452906+0000/76.306472778s send_stamp 78.287384033s delta_ub 1.980909944s) -- 0x3e02180 con 0x396a400 2026-02-11T23:31:25.452+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:25.452+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:25.452+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:25.452+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 23 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:25.452906+0000/76.306472778s send_stamp 76.306472778s delta_ub -1.980817080s) ==== 2033+0+0 (crc 0 0 0) 0x3da3680 con 0x396a000 2026-02-11T23:31:25.452+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 updating session's projected_epoch from 117 to ping map epoch of 118 2026-02-11T23:31:25.452+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:25.452+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980924010s,-1.980817080s]) 2026-02-11T23:31:25.452+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:25.452906+0000/76.306472778s send_stamp 78.287399292s delta_ub 1.980924010s) -- 0x3da2480 con 0x396a000 2026-02-11T23:31:25.452+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:25.452+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:25.452+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:25.452+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:25.520+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 19 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:25.520821+0000/73.706214905s send_stamp 73.706214905s delta_ub -4.648911476s) ==== 2033+0+0 (crc 0 0 0) 0x3da2480 con 0x3f56000 2026-02-11T23:31:25.520+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649074078s,-4.648911476s]) 2026-02-11T23:31:25.520+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:25.520821+0000/73.706214905s send_stamp 78.355293274s delta_ub 4.649074078s) -- 0x3da3680 con 0x3f56000 2026-02-11T23:31:25.520+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:25.520+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:25.520+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:25.520+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 updating session's projected_epoch from 117 to ping map epoch of 118 2026-02-11T23:31:25.520+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:25.520+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 19 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:25.520821+0000/73.706214905s send_stamp 73.706214905s delta_ub -4.648911476s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x3f56400 2026-02-11T23:31:25.520+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649093151s,-4.648911476s]) 2026-02-11T23:31:25.520+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:25.520821+0000/73.706214905s send_stamp 78.355308533s delta_ub 4.649093151s) -- 0x3b1bb00 con 0x3f56400 2026-02-11T23:31:25.520+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:25.520+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:25.520+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:25.520+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:25.595+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1249 ==== pg_lease(1.0 pg_lease(ru 84.448936462s ub 92.449028015s int 16.000000000s) e118/118) ==== 58+0+0 (unknown 0 0 0) 0x3b1bb00 con 0x3752000 2026-02-11T23:31:25.595+0000 7fac703fa640 15 osd.0 118 enqueue_peering_evt 1.0 epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 84.448936462s ub 92.449028015s int 16.000000000s) 2026-02-11T23:31:25.595+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 84.448936462s ub 92.449028015s int 16.000000000s)) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 84.448936462s ub 92.449028015s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 84.448936462s ub 92.449028015s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) pg 0x4a2a000 2026-02-11T23:31:25.595+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 84.448936462s ub 92.449028015s int 16.000000000s) 2026-02-11T23:31:25.595+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:25.595+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 84.448936462s ub 92.449028015s int 16.000000000s) 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -1.980817080s -> ru 86.429756165s 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until now 86.429756165s 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.980924010s -> ruub 94.429954529s 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 94.429954529s 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:25.595+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_lease_ack(1.0 pg_lease_ack(ruub 92.449028015s) e118/118) -- 0x5612b00 con 0x3752000 2026-02-11T23:31:25.595+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:25.595+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:25.897+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1257 ==== pg_lease(4.0 pg_lease(ru 76.844993591s ub 84.845077515s int 16.000000000s) e118/118) ==== 58+0+0 (unknown 0 0 0) 0x3b1bb00 con 0x3e4a000 2026-02-11T23:31:25.897+0000 7fac703fa640 15 osd.0 118 enqueue_peering_evt 4.0 epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 76.844993591s ub 84.845077515s int 16.000000000s) 2026-02-11T23:31:25.897+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 76.844993591s ub 84.845077515s int 16.000000000s)) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 76.844993591s ub 84.845077515s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 76.844993591s ub 84.845077515s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) pg 0x4859000 2026-02-11T23:31:25.897+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 76.844993591s ub 84.845077515s int 16.000000000s) 2026-02-11T23:31:25.897+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:25.897+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 76.844993591s ub 84.845077515s int 16.000000000s) 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.886796951s -> ru 86.731788635s 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until now 86.731788635s 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.887121201s -> ruub 94.732200623s 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 94.732200623s 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:25.897+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_lease_ack(4.0 pg_lease_ack(ruub 84.845077515s) e118/118) -- 0x5612c60 con 0x3e4a000 2026-02-11T23:31:25.897+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:25.897+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:26.052+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:26.052+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:26.141+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:26.141+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:26.141+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:26.141+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:26.141+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:26.141+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.072 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:26.141+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:26.141, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:26.141+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:26.141+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:26.141+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:26.141+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:26.141+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:26.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:26.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:26.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:56.171889+0000) 2026-02-11T23:31:26.759+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 23 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:26.759752+0000/69.707260132s send_stamp 69.707260132s delta_ub -9.886865616s) ==== 2033+0+0 (crc 0 0 0) 0x3b1bb00 con 0x3f57400 2026-02-11T23:31:26.759+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886960030s,-9.886865616s]) 2026-02-11T23:31:26.759+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 23 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:26.759752+0000/69.707260132s send_stamp 69.707260132s delta_ub -9.886865616s) ==== 2033+0+0 (crc 0 0 0) 0x3e02180 con 0x3f57800 2026-02-11T23:31:26.759+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:26.759752+0000/69.707260132s send_stamp 79.594215393s delta_ub 9.886960030s) -- 0x3b1b380 con 0x3f57400 2026-02-11T23:31:26.759+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:26.759+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:26.759+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:26.759+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:26.759+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886975288s,-9.886865616s]) 2026-02-11T23:31:26.759+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:26.759752+0000/69.707260132s send_stamp 79.594230652s delta_ub 9.886975288s) -- 0x3e02300 con 0x3f57800 2026-02-11T23:31:26.759+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:26.759+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:26.759+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:26.759+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.027+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:27.027+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:27.150+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:27.150+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:27.150+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:27.150+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:27.150+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:27.150+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.072 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:27.150+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:27.150, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:27.150+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:27.150+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:27.150+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:27.150+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:27.150+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:27.152+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 24 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:27.153073+0000/78.006637573s send_stamp 78.006637573s delta_ub -1.980792761s) ==== 2033+0+0 (crc 0 0 0) 0x3e02300 con 0x396a400 2026-02-11T23:31:27.152+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980897665s,-1.980792761s]) 2026-02-11T23:31:27.152+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:27.153073+0000/78.006637573s send_stamp 79.987533569s delta_ub 1.980897665s) -- 0x3e02180 con 0x396a400 2026-02-11T23:31:27.152+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:27.152+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.152+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.152+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 24 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:27.153073+0000/78.006637573s send_stamp 78.006637573s delta_ub -1.980792761s) ==== 2033+0+0 (crc 0 0 0) 0x3da3680 con 0x396a000 2026-02-11T23:31:27.152+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.152+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980910420s,-1.980792761s]) 2026-02-11T23:31:27.152+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:27.153073+0000/78.006637573s send_stamp 79.987548828s delta_ub 1.980910420s) -- 0x3da2480 con 0x396a000 2026-02-11T23:31:27.152+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:27.152+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.152+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.152+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:27.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:27.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:57.171910+0000) 2026-02-11T23:31:27.820+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 20 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:27.821001+0000/76.006401062s send_stamp 76.006401062s delta_ub -4.648978233s) ==== 2033+0+0 (crc 0 0 0) 0x3da2480 con 0x3f56000 2026-02-11T23:31:27.820+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649076462s,-4.648978233s]) 2026-02-11T23:31:27.820+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:27.821001+0000/76.006401062s send_stamp 80.655471802s delta_ub 4.649076462s) -- 0x3da3680 con 0x3f56000 2026-02-11T23:31:27.820+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:27.820+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.820+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.820+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 20 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:27.821001+0000/76.006401062s send_stamp 76.006401062s delta_ub -4.648978233s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b380 con 0x3f56400 2026-02-11T23:31:27.820+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.820+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649092674s,-4.648978233s]) 2026-02-11T23:31:27.820+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:27.821001+0000/76.006401062s send_stamp 80.655494690s delta_ub 4.649092674s) -- 0x3b1bb00 con 0x3f56400 2026-02-11T23:31:27.820+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:27.820+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.820+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.820+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.973+0000 7fac4d160640 5 osd.0 118 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x537bf3/0x5de000, compress 0x0/0x0/0x0, omap 0xc287, meta 0x1a43d79), peers [1,2,3,4] op hist []) 2026-02-11T23:31:27.973+0000 7fac4d160640 20 osd.0 118 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:27.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 80.807754517s delta_ub 1.980910420s) -- 0x3b1a780 con 0x396b400 2026-02-11T23:31:27.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 80.807754517s delta_ub 1.980910420s) -- 0x3b1b980 con 0x3752400 2026-02-11T23:31:27.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 80.807754517s delta_ub 4.649092674s) -- 0x3b1a600 con 0x396b800 2026-02-11T23:31:27.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 80.807754517s delta_ub 4.649092674s) -- 0x567c000 con 0x34c1000 2026-02-11T23:31:27.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 80.807754517s delta_ub 7.310456276s) -- 0x567c180 con 0x3740000 2026-02-11T23:31:27.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 80.807754517s delta_ub 7.310456276s) -- 0x567c300 con 0x3740400 2026-02-11T23:31:27.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 80.807754517s delta_ub 9.886975288s) -- 0x567c480 con 0x3740800 2026-02-11T23:31:27.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 80.807754517s delta_ub 9.886975288s) -- 0x567c600 con 0x3740c00 2026-02-11T23:31:27.973+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 25 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 78.826965332s delta_ub -1.980796695s) ==== 2033+0+0 (crc 0 0 0) 0x567c600 con 0x396b400 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.973+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 24 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 76.158805847s delta_ub -4.648952961s) ==== 2033+0+0 (crc 0 0 0) 0x3e03680 con 0x396b800 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.973+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980915666s,-1.980796695s]) 2026-02-11T23:31:27.973+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 25 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 78.826988220s delta_ub -1.980764508s) ==== 2033+0+0 (crc 0 0 0) 0x567c300 con 0x3752400 2026-02-11T23:31:27.973+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:27.973+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.973+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.973+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.973+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980893731s,-1.980764508s]) 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.973+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 24 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 76.158821106s delta_ub -4.648934364s) ==== 2033+0+0 (crc 0 0 0) 0x567c300 con 0x34c1000 2026-02-11T23:31:27.973+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649092674s,-4.648952961s]) 2026-02-11T23:31:27.973+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 22 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 73.497482300s delta_ub -7.310272694s) ==== 2033+0+0 (crc 0 0 0) 0x567c600 con 0x3740000 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.973+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310419559s,-7.310272694s]) 2026-02-11T23:31:27.973+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:27.973+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 22 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 73.497467041s delta_ub -7.310286999s) ==== 2033+0+0 (crc 0 0 0) 0x3e03680 con 0x3740400 2026-02-11T23:31:27.973+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.973+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.973+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.973+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649089813s,-4.648934364s]) 2026-02-11T23:31:27.973+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 22 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 70.920967102s delta_ub -9.886788368s) ==== 2033+0+0 (crc 0 0 0) 0x567c600 con 0x3740c00 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.973+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310451508s,-7.310286999s]) 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.973+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.973+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886969566s,-9.886788368s]) 2026-02-11T23:31:27.973+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 22 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:27.973340+0000/80.807754517s send_stamp 70.920951843s delta_ub -9.886803627s) ==== 2033+0+0 (crc 0 0 0) 0x3e03680 con 0x3740800 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:27.973+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:27.973+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886990547s,-9.886803627s]) 2026-02-11T23:31:28.014+0000 7fac60ee5640 20 osd.0 118 reports for 0 queries 2026-02-11T23:31:28.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4ff0000 con 0x396a800 2026-02-11T23:31:28.014+0000 7fac60ee5640 15 osd.0 118 collect_pg_stats 2026-02-11T23:31:28.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836501 v 0) -- 0x567e000 con 0x396a800 2026-02-11T23:31:28.021+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:28.021+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:28.109+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:28.109+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:28.109+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:28.109+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:28.109+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:28.109+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.072 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:28.109+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:28.109, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:28.109+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:28.109+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:28.109+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:28.109+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:28.109+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:28.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:28.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:28.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:58.171934+0000) 2026-02-11T23:31:29.054+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:29.054+0000 7fac68beb640 10 osd.0 118 maybe_update_heartbeat_peers forcing update after 20.013037 seconds 2026-02-11T23:31:29.054+0000 7fac68beb640 10 osd.0 118 reset_heartbeat_peers 2026-02-11T23:31:29.054+0000 7fac68beb640 10 osd.0 118 maybe_update_heartbeat_peers updating 2026-02-11T23:31:29.054+0000 7fac68beb640 10 osd.0 118 adding neighbor peer osd.1 2026-02-11T23:31:29.054+0000 7fac68beb640 10 osd.0 118 adding neighbor peer osd.4 2026-02-11T23:31:29.054+0000 7fac68beb640 10 osd.0 118 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:31:29.054+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:29.095+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:29.095+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:29.095+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:29.095+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:29.095+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:29.095+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.072 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:29.095+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:29.095, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:29.095+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:29.095+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:29.095+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:29.095+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:29.095+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:29.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:29.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:29.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:30:59.171958+0000) 2026-02-11T23:31:29.483+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 24 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:29.483411+0000/75.007453918s send_stamp 75.007453918s delta_ub -7.310272694s) ==== 2033+0+0 (crc 0 0 0) 0x3e03680 con 0x3f56c00 2026-02-11T23:31:29.483+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310433865s,-7.310272694s]) 2026-02-11T23:31:29.483+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 24 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:29.483411+0000/75.007453918s send_stamp 75.007453918s delta_ub -7.310272694s) ==== 2033+0+0 (crc 0 0 0) 0x567c300 con 0x3f56800 2026-02-11T23:31:29.483+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:29.483411+0000/75.007453918s send_stamp 82.317886353s delta_ub 7.310433865s) -- 0x3e03380 con 0x3f56c00 2026-02-11T23:31:29.483+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:29.483+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:29.483+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:29.483+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:29.483+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310453415s,-7.310272694s]) 2026-02-11T23:31:29.483+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:29.483411+0000/75.007453918s send_stamp 82.317901611s delta_ub 7.310453415s) -- 0x567c480 con 0x3f56800 2026-02-11T23:31:29.483+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:29.483+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:29.483+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:29.483+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:29.659+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 24 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:29.659880+0000/72.607383728s send_stamp 72.607383728s delta_ub -9.886788368s) ==== 2033+0+0 (crc 0 0 0) 0x567c480 con 0x3f57800 2026-02-11T23:31:29.659+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886955261s,-9.886788368s]) 2026-02-11T23:31:29.659+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:29.659880+0000/72.607383728s send_stamp 82.494338989s delta_ub 9.886955261s) -- 0x567c300 con 0x3f57800 2026-02-11T23:31:29.659+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:29.659+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:29.659+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:29.659+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:29.659+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 24 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:29.659880+0000/72.607383728s send_stamp 72.607383728s delta_ub -9.886788368s) ==== 2033+0+0 (crc 0 0 0) 0x567c600 con 0x3f57400 2026-02-11T23:31:29.659+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886966705s,-9.886788368s]) 2026-02-11T23:31:29.659+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:29.659880+0000/72.607383728s send_stamp 82.494354248s delta_ub 9.886966705s) -- 0x567c180 con 0x3f57400 2026-02-11T23:31:29.659+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:29.659+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:29.659+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:29.659+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:30.086+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:30.086+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:30.125+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:30.125+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:30.125+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:30.125+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.066 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:30.125+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:30.125, recovery is active?:false restrictions:<....> 2026-02-11T23:31:30.125+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:30.125+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:30.125+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:30.125+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:30.125+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:30.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:30.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:30.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:00.171984+0000) 2026-02-11T23:31:30.171+0000 7fac616e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:30.273+0000 7fac4d160640 5 osd.0 118 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x537bf3/0x5de000, compress 0x0/0x0/0x0, omap 0xc287, meta 0x1a43d79), peers [1,2,3,4] op hist []) 2026-02-11T23:31:30.273+0000 7fac4d160640 20 osd.0 118 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:30.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 83.107925415s delta_ub 1.980893731s) -- 0x567c780 con 0x396b400 2026-02-11T23:31:30.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 83.107925415s delta_ub 1.980893731s) -- 0x567c900 con 0x3752400 2026-02-11T23:31:30.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 83.107925415s delta_ub 4.649089813s) -- 0x567ca80 con 0x396b800 2026-02-11T23:31:30.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 83.107925415s delta_ub 4.649089813s) -- 0x567cc00 con 0x34c1000 2026-02-11T23:31:30.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 83.107925415s delta_ub 7.310453415s) -- 0x567cd80 con 0x3740000 2026-02-11T23:31:30.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 83.107925415s delta_ub 7.310453415s) -- 0x567cf00 con 0x3740400 2026-02-11T23:31:30.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 83.107925415s delta_ub 9.886966705s) -- 0x567d080 con 0x3740800 2026-02-11T23:31:30.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 83.107925415s delta_ub 9.886966705s) -- 0x567d200 con 0x3740c00 2026-02-11T23:31:30.273+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 26 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 81.127113342s delta_ub -1.980808973s) ==== 2033+0+0 (crc 0 0 0) 0x567cd80 con 0x396b400 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:30.273+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980898023s,-1.980808973s]) 2026-02-11T23:31:30.273+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 26 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 81.127128601s delta_ub -1.980792165s) ==== 2033+0+0 (crc 0 0 0) 0x3e03080 con 0x3752400 2026-02-11T23:31:30.273+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:30.273+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:30.273+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:30.273+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:30.273+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980894923s,-1.980792165s]) 2026-02-11T23:31:30.273+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 25 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 78.458969116s delta_ub -4.648946285s) ==== 2033+0+0 (crc 0 0 0) 0x567d080 con 0x396b800 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:30.273+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649063587s,-4.648946285s]) 2026-02-11T23:31:30.273+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 25 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 78.458984375s delta_ub -4.648929596s) ==== 2033+0+0 (crc 0 0 0) 0x3e03080 con 0x34c1000 2026-02-11T23:31:30.273+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:30.273+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:30.273+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:30.273+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:30.273+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649053097s,-4.648929596s]) 2026-02-11T23:31:30.273+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 23 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 73.221099854s delta_ub -9.886818886s) ==== 2033+0+0 (crc 0 0 0) 0x567d080 con 0x3740800 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:30.273+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886951447s,-9.886818886s]) 2026-02-11T23:31:30.273+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 23 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 75.797645569s delta_ub -7.310274601s) ==== 2033+0+0 (crc 0 0 0) 0x567d200 con 0x3740000 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:30.273+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 23 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 75.797630310s delta_ub -7.310294628s) ==== 2033+0+0 (crc 0 0 0) 0x567d080 con 0x3740400 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:30.273+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310415268s,-7.310274601s]) 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:30.273+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:30.273+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310441017s,-7.310294628s]) 2026-02-11T23:31:30.273+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 23 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:30.273504+0000/83.107925415s send_stamp 73.221153259s delta_ub -9.886762619s) ==== 2033+0+0 (crc 0 0 0) 0x567d200 con 0x3740c00 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:30.273+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:30.273+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886935234s,-9.886762619s]) 2026-02-11T23:31:31.088+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:31.088+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:31.104+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:31.104+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:31.104+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:31.104+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:31.104+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:31.104+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.066 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:31.104+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:31.104, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:31.104+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:31.104+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:31.104+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:31.104+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:31.104+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:31.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:31.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:31.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:01.172018+0000) 2026-02-11T23:31:31.853+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 25 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:31.853229+0000/82.706794739s send_stamp 82.706794739s delta_ub -1.980792165s) ==== 2033+0+0 (crc 0 0 0) 0x3e03080 con 0x396a400 2026-02-11T23:31:31.853+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980911016s,-1.980792165s]) 2026-02-11T23:31:31.853+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:31.853229+0000/82.706794739s send_stamp 84.687705994s delta_ub 1.980911016s) -- 0x3e02000 con 0x396a400 2026-02-11T23:31:31.853+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 25 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:31.853229+0000/82.706794739s send_stamp 82.706794739s delta_ub -1.980792165s) ==== 2033+0+0 (crc 0 0 0) 0x567d080 con 0x396a000 2026-02-11T23:31:31.853+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:31.853+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:31.853+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:31.853+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:31.853+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980930448s,-1.980792165s]) 2026-02-11T23:31:31.853+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:31.853229+0000/82.706794739s send_stamp 84.687721252s delta_ub 1.980930448s) -- 0x567ca80 con 0x396a000 2026-02-11T23:31:31.853+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:31.853+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:31.853+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:31.853+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:32.042+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:32.042+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:32.103+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:32.103+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:32.103+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:32.103+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.066 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:32.103+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:32.103, recovery is active?:false restrictions:<....> 2026-02-11T23:31:32.103+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:32.103+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:32.103+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:32.103+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:32.103+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:32.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:32.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:32.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:02.172043+0000) 2026-02-11T23:31:33.014+0000 7fac60ee5640 20 osd.0 118 reports for 0 queries 2026-02-11T23:31:33.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4ff0380 con 0x396a800 2026-02-11T23:31:33.014+0000 7fac60ee5640 15 osd.0 118 collect_pg_stats 2026-02-11T23:31:33.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836502 v 0) -- 0x567e300 con 0x396a800 2026-02-11T23:31:33.016+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:33.016+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:33.102+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:33.102+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:33.102+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:33.102+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.066 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:33.102+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:33.103, recovery is active?:false restrictions:<....> 2026-02-11T23:31:33.102+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:33.102+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:33.102+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:33.102+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:33.102+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:33.120+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 21 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:33.121184+0000/81.306579590s send_stamp 81.306579590s delta_ub -4.648929596s) ==== 2033+0+0 (crc 0 0 0) 0x567ca80 con 0x3f56000 2026-02-11T23:31:33.120+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649076939s,-4.648929596s]) 2026-02-11T23:31:33.120+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 21 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:33.121184+0000/81.306579590s send_stamp 81.306579590s delta_ub -4.648929596s) ==== 2033+0+0 (crc 0 0 0) 0x567d200 con 0x3f56400 2026-02-11T23:31:33.120+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:33.121184+0000/81.306579590s send_stamp 85.955657959s delta_ub 4.649076939s) -- 0x567d080 con 0x3f56000 2026-02-11T23:31:33.120+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:33.120+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:33.120+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:33.120+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:33.120+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649092197s,-4.648929596s]) 2026-02-11T23:31:33.120+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:33.121184+0000/81.306579590s send_stamp 85.955673218s delta_ub 4.649092197s) -- 0x567cd80 con 0x3f56400 2026-02-11T23:31:33.120+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:33.120+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:33.120+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:33.120+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:33.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:33.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:33.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:03.172068+0000) 2026-02-11T23:31:33.595+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1250 ==== pg_lease(1.0 pg_lease(ru 92.449028015s ub 100.449119568s int 16.000000000s) e118/118) ==== 58+0+0 (unknown 0 0 0) 0x567cd80 con 0x3752000 2026-02-11T23:31:33.595+0000 7fac703fa640 15 osd.0 118 enqueue_peering_evt 1.0 epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 92.449028015s ub 100.449119568s int 16.000000000s) 2026-02-11T23:31:33.595+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 92.449028015s ub 100.449119568s int 16.000000000s)) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 92.449028015s ub 100.449119568s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 92.449028015s ub 100.449119568s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) pg 0x4a2a000 2026-02-11T23:31:33.595+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:33.595+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 92.449028015s ub 100.449119568s int 16.000000000s) 2026-02-11T23:31:33.595+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 92.449028015s ub 100.449119568s int 16.000000000s) 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -1.980792165s -> ru 94.429824829s 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until now 94.429824829s 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.980930448s -> ruub 102.430046082s 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 102.430046082s 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:33.595+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_lease_ack(1.0 pg_lease_ack(ruub 100.449119568s) e118/118) -- 0x5612dc0 con 0x3752000 2026-02-11T23:31:33.595+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:33.595+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:33.897+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1258 ==== pg_lease(4.0 pg_lease(ru 84.845077515s ub 92.845146179s int 16.000000000s) e118/118) ==== 58+0+0 (unknown 0 0 0) 0x567cd80 con 0x3e4a000 2026-02-11T23:31:33.897+0000 7fac703fa640 15 osd.0 118 enqueue_peering_evt 4.0 epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 84.845077515s ub 92.845146179s int 16.000000000s) 2026-02-11T23:31:33.897+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 84.845077515s ub 92.845146179s int 16.000000000s)) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 84.845077515s ub 92.845146179s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 84.845077515s ub 92.845146179s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) pg 0x4859000 2026-02-11T23:31:33.897+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:33.897+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 84.845077515s ub 92.845146179s int 16.000000000s) 2026-02-11T23:31:33.897+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 84.845077515s ub 92.845146179s int 16.000000000s) 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.886762619s -> ru 94.731834412s 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until now 94.731834412s 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.886935234s -> ruub 102.732078552s 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 102.732078552s 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:33.897+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_lease_ack(4.0 pg_lease_ack(ruub 92.845146179s) e118/118) -- 0x5612f20 con 0x3e4a000 2026-02-11T23:31:33.897+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:33.897+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:33.976+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:33.976+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:34.063+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:34.063+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:34.063+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:34.063+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:34.063+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:34.063+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.066 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:34.063+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:34.063, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:34.063+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:34.063+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:34.063+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:34.063+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:34.063+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:34.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:34.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:34.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:04.172091+0000) 2026-02-11T23:31:34.359+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 25 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:34.360034+0000/77.307540894s send_stamp 77.307540894s delta_ub -9.886762619s) ==== 2033+0+0 (crc 0 0 0) 0x567cd80 con 0x3f57400 2026-02-11T23:31:34.359+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886966705s,-9.886762619s]) 2026-02-11T23:31:34.359+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 25 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:34.360034+0000/77.307540894s send_stamp 77.307540894s delta_ub -9.886762619s) ==== 2033+0+0 (crc 0 0 0) 0x3e02000 con 0x3f57800 2026-02-11T23:31:34.359+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:34.360034+0000/77.307540894s send_stamp 87.194511414s delta_ub 9.886966705s) -- 0x567d200 con 0x3f57400 2026-02-11T23:31:34.359+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:34.359+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.359+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.359+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.359+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886982918s,-9.886762619s]) 2026-02-11T23:31:34.359+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:34.360034+0000/77.307540894s send_stamp 87.194526672s delta_ub 9.886982918s) -- 0x3e03080 con 0x3f57800 2026-02-11T23:31:34.359+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:34.359+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.359+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.359+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.373+0000 7fac4d160640 5 osd.0 118 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x537bf3/0x5de000, compress 0x0/0x0/0x0, omap 0xc287, meta 0x1a43d79), peers [1,2,3,4] op hist []) 2026-02-11T23:31:34.373+0000 7fac4d160640 20 osd.0 118 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:34.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 87.208068848s delta_ub 1.980930448s) -- 0x567d380 con 0x396b400 2026-02-11T23:31:34.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 87.208068848s delta_ub 1.980930448s) -- 0x567d500 con 0x3752400 2026-02-11T23:31:34.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 87.208068848s delta_ub 4.649092197s) -- 0x567d680 con 0x396b800 2026-02-11T23:31:34.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 87.208068848s delta_ub 4.649092197s) -- 0x567d800 con 0x34c1000 2026-02-11T23:31:34.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 87.208068848s delta_ub 7.310441017s) -- 0x567d980 con 0x3740000 2026-02-11T23:31:34.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 87.208068848s delta_ub 7.310441017s) -- 0x567db00 con 0x3740400 2026-02-11T23:31:34.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 87.208068848s delta_ub 9.886982918s) -- 0x567dc80 con 0x3740800 2026-02-11T23:31:34.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 87.208068848s delta_ub 9.886982918s) -- 0x567de00 con 0x3740c00 2026-02-11T23:31:34.373+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 27 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 85.227256775s delta_ub -1.980812430s) ==== 2033+0+0 (crc 0 0 0) 0x567d980 con 0x396b400 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.373+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980891109s,-1.980812430s]) 2026-02-11T23:31:34.373+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 26 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 82.559097290s delta_ub -4.648972511s) ==== 2033+0+0 (crc 0 0 0) 0x3919200 con 0x34c1000 2026-02-11T23:31:34.373+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:34.373+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.373+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.373+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.373+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649057388s,-4.648972511s]) 2026-02-11T23:31:34.373+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 26 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 82.559104919s delta_ub -4.648961067s) ==== 2033+0+0 (crc 0 0 0) 0x567dc80 con 0x396b800 2026-02-11T23:31:34.373+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 27 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 85.227272034s delta_ub -1.980796337s) ==== 2033+0+0 (crc 0 0 0) 0x3919200 con 0x3752400 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.373+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649057388s,-4.648961067s]) 2026-02-11T23:31:34.373+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 24 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 79.897766113s delta_ub -7.310305119s) ==== 2033+0+0 (crc 0 0 0) 0x567de00 con 0x3740000 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.373+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310412407s,-7.310305119s]) 2026-02-11T23:31:34.373+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 24 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 77.321250916s delta_ub -9.886820793s) ==== 2033+0+0 (crc 0 0 0) 0x567dc80 con 0x3740800 2026-02-11T23:31:34.373+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:34.373+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.373+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.373+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.373+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980909109s,-1.980796337s]) 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.373+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886941910s,-9.886820793s]) 2026-02-11T23:31:34.373+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 24 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 79.897781372s delta_ub -7.310290337s) ==== 2033+0+0 (crc 0 0 0) 0x567dc80 con 0x3740400 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.373+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 24 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:34.373651+0000/87.208068848s send_stamp 77.321266174s delta_ub -9.886798859s) ==== 2033+0+0 (crc 0 0 0) 0x567de00 con 0x3740c00 2026-02-11T23:31:34.373+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.373+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310425758s,-7.310290337s]) 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.373+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.373+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886940956s,-9.886798859s]) 2026-02-11T23:31:34.783+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 25 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:34.783536+0000/80.307571411s send_stamp 80.307571411s delta_ub -7.310290337s) ==== 2033+0+0 (crc 0 0 0) 0x3919200 con 0x3f56800 2026-02-11T23:31:34.783+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310427189s,-7.310290337s]) 2026-02-11T23:31:34.783+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 25 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:34.783536+0000/80.307571411s send_stamp 80.307571411s delta_ub -7.310290337s) ==== 2033+0+0 (crc 0 0 0) 0x567dc80 con 0x3f56c00 2026-02-11T23:31:34.783+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:34.783536+0000/80.307571411s send_stamp 87.618003845s delta_ub 7.310427189s) -- 0x3da3380 con 0x3f56800 2026-02-11T23:31:34.783+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:34.783+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.783+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.783+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.783+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310439110s,-7.310290337s]) 2026-02-11T23:31:34.783+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:34.783536+0000/80.307571411s send_stamp 87.618011475s delta_ub 7.310439110s) -- 0x567db00 con 0x3f56c00 2026-02-11T23:31:34.783+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:34.783+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:34.783+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:34.783+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:34.950+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:34.950+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:35.102+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:35.102+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:35.102+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:35.102+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:35.102+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:35.102+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.060 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:35.102+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:35.102, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:35.102+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:35.102+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:35.102+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:35.102+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:35.102+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:35.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:35.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:35.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:05.172116+0000) 2026-02-11T23:31:35.966+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:35.966+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:36.107+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:36.107+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:36.107+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:36.107+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.060 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:36.107+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:36.107, recovery is active?:false restrictions:<....> 2026-02-11T23:31:36.107+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:36.107+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:36.107+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:36.107+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:36.107+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:36.172+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:36.172+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:36.172+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:06.172142+0000) 2026-02-11T23:31:36.621+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 22 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:36.621361+0000/84.806755066s send_stamp 84.806755066s delta_ub -4.648961067s) ==== 2033+0+0 (crc 0 0 0) 0x567db00 con 0x3f56000 2026-02-11T23:31:36.621+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649115562s,-4.648961067s]) 2026-02-11T23:31:36.621+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 22 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:36.621361+0000/84.806755066s send_stamp 84.806755066s delta_ub -4.648961067s) ==== 2033+0+0 (crc 0 0 0) 0x567de00 con 0x3f56400 2026-02-11T23:31:36.621+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:36.621361+0000/84.806755066s send_stamp 89.455871582s delta_ub 4.649115562s) -- 0x567dc80 con 0x3f56000 2026-02-11T23:31:36.621+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:36.621+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:36.621+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:36.621+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:36.621+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649131298s,-4.648961067s]) 2026-02-11T23:31:36.621+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:36.621361+0000/84.806755066s send_stamp 89.455886841s delta_ub 4.649131298s) -- 0x567d980 con 0x3f56400 2026-02-11T23:31:36.621+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:36.621+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:36.621+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:36.621+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:36.922+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:36.922+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:37.122+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:37.122+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:37.122+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:37.122+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.060 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:37.122+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:37.122, recovery is active?:false restrictions:<....> 2026-02-11T23:31:37.122+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:37.122+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:37.122+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:37.122+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:37.122+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:37.153+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 26 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:37.153387+0000/88.006950378s send_stamp 88.006950378s delta_ub -1.980796337s) ==== 2033+0+0 (crc 0 0 0) 0x3da3380 con 0x396a400 2026-02-11T23:31:37.153+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980909705s,-1.980796337s]) 2026-02-11T23:31:37.153+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:37.153387+0000/88.006950378s send_stamp 89.987861633s delta_ub 1.980909705s) -- 0x3919200 con 0x396a400 2026-02-11T23:31:37.153+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 26 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:37.153387+0000/88.006950378s send_stamp 88.006950378s delta_ub -1.980796337s) ==== 2033+0+0 (crc 0 0 0) 0x567dc80 con 0x396a000 2026-02-11T23:31:37.153+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:37.153+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:37.153+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:37.153+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:37.153+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980924249s,-1.980796337s]) 2026-02-11T23:31:37.153+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:37.153387+0000/88.006950378s send_stamp 89.987876892s delta_ub 1.980924249s) -- 0x567db00 con 0x396a000 2026-02-11T23:31:37.153+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:37.153+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:37.153+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:37.153+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:37.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:37.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:37.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:07.172167+0000) 2026-02-11T23:31:37.884+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:37.884+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:38.014+0000 7fac60ee5640 20 osd.0 118 reports for 0 queries 2026-02-11T23:31:38.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4ff0700 con 0x396a800 2026-02-11T23:31:38.014+0000 7fac60ee5640 15 osd.0 118 collect_pg_stats 2026-02-11T23:31:38.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836503 v 0) -- 0x567e600 con 0x396a800 2026-02-11T23:31:38.171+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:38.171+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:38.171+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:38.171+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:38.171+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:38.171+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.060 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:38.171+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:38.171, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:38.171+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:38.171+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:38.171+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:38.171+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:38.171+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:38.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:38.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:38.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:08.172191+0000) 2026-02-11T23:31:38.253+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 27 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:38.253568+0000/89.107139587s send_stamp 89.107139587s delta_ub -1.980802298s) ==== 2033+0+0 (crc 0 0 0) 0x3919200 con 0x396a400 2026-02-11T23:31:38.253+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 27 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:38.253568+0000/89.107139587s send_stamp 89.107139587s delta_ub -1.980802298s) ==== 2033+0+0 (crc 0 0 0) 0x567db00 con 0x396a000 2026-02-11T23:31:38.253+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980907679s,-1.980802298s]) 2026-02-11T23:31:38.253+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:38.253568+0000/89.107139587s send_stamp 91.088043213s delta_ub 1.980907679s) -- 0x3da3380 con 0x396a400 2026-02-11T23:31:38.253+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:38.253+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.253+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.253+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.253+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980926514s,-1.980802298s]) 2026-02-11T23:31:38.253+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:38.253568+0000/89.107139587s send_stamp 91.088058472s delta_ub 1.980926514s) -- 0x567dc80 con 0x396a000 2026-02-11T23:31:38.253+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:38.253+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.253+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.253+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.321+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 23 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:38.321600+0000/86.506996155s send_stamp 86.506996155s delta_ub -4.648946762s) ==== 2033+0+0 (crc 0 0 0) 0x567dc80 con 0x3f56000 2026-02-11T23:31:38.321+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649064064s,-4.648946762s]) 2026-02-11T23:31:38.321+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 23 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:38.321600+0000/86.506996155s send_stamp 86.506996155s delta_ub -4.648946762s) ==== 2033+0+0 (crc 0 0 0) 0x567d980 con 0x3f56400 2026-02-11T23:31:38.321+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:38.321600+0000/86.506996155s send_stamp 91.156059265s delta_ub 4.649064064s) -- 0x567db00 con 0x3f56000 2026-02-11T23:31:38.321+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:38.321+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.321+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.321+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.321+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649081707s,-4.648946762s]) 2026-02-11T23:31:38.321+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:38.321600+0000/86.506996155s send_stamp 91.156082153s delta_ub 4.649081707s) -- 0x567de00 con 0x3f56400 2026-02-11T23:31:38.321+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:38.321+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.321+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.321+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.460+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 26 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:38.460187+0000/81.407699585s send_stamp 81.407699585s delta_ub -9.886798859s) ==== 2033+0+0 (crc 0 0 0) 0x567de00 con 0x3f57400 2026-02-11T23:31:38.460+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886947632s,-9.886798859s]) 2026-02-11T23:31:38.460+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 26 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:38.460187+0000/81.407699585s send_stamp 81.407699585s delta_ub -9.886798859s) ==== 2033+0+0 (crc 0 0 0) 0x3da3380 con 0x3f57800 2026-02-11T23:31:38.460+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:38.460187+0000/81.407699585s send_stamp 91.294639587s delta_ub 9.886947632s) -- 0x567d980 con 0x3f57400 2026-02-11T23:31:38.460+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:38.460+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.460+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.460+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.460+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886956215s,-9.886798859s]) 2026-02-11T23:31:38.460+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:38.460187+0000/81.407699585s send_stamp 91.294654846s delta_ub 9.886956215s) -- 0x3919200 con 0x3f57800 2026-02-11T23:31:38.460+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:38.460+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.460+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.460+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.473+0000 7fac4d160640 5 osd.0 118 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x537bf3/0x5de000, compress 0x0/0x0/0x0, omap 0xc287, meta 0x1a43d79), peers [1,2,3,4] op hist []) 2026-02-11T23:31:38.473+0000 7fac4d160640 20 osd.0 118 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:38.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 91.308219910s delta_ub 1.980926514s) -- 0x567cc00 con 0x396b400 2026-02-11T23:31:38.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 91.308219910s delta_ub 1.980926514s) -- 0x567c900 con 0x3752400 2026-02-11T23:31:38.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 91.308219910s delta_ub 4.649081707s) -- 0x567c300 con 0x396b800 2026-02-11T23:31:38.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 91.308219910s delta_ub 4.649081707s) -- 0x567c480 con 0x34c1000 2026-02-11T23:31:38.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 91.308219910s delta_ub 7.310439110s) -- 0x567c000 con 0x3740000 2026-02-11T23:31:38.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 91.308219910s delta_ub 7.310439110s) -- 0x567cf00 con 0x3740400 2026-02-11T23:31:38.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 91.308219910s delta_ub 9.886956215s) -- 0x3e02480 con 0x3740800 2026-02-11T23:31:38.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 91.308219910s delta_ub 9.886956215s) -- 0x3e03980 con 0x3740c00 2026-02-11T23:31:38.473+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 28 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 89.327407837s delta_ub -1.980806947s) ==== 2033+0+0 (crc 0 0 0) 0x3e03980 con 0x396b400 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.473+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980889440s,-1.980806947s]) 2026-02-11T23:31:38.473+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 28 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 89.327430725s delta_ub -1.980792522s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b980 con 0x3752400 2026-02-11T23:31:38.473+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:38.473+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.473+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.473+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.473+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980890274s,-1.980792522s]) 2026-02-11T23:31:38.473+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 25 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 83.997909546s delta_ub -7.310305595s) ==== 2033+0+0 (crc 0 0 0) 0x3e03980 con 0x3740000 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.473+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310417175s,-7.310305595s]) 2026-02-11T23:31:38.473+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 27 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 86.659263611s delta_ub -4.648958683s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b980 con 0x34c1000 2026-02-11T23:31:38.473+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:38.473+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.473+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.473+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.473+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649076462s,-4.648958683s]) 2026-02-11T23:31:38.473+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 27 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 86.659278870s delta_ub -4.648939133s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x396b800 2026-02-11T23:31:38.473+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 25 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 81.421394348s delta_ub -9.886825562s) ==== 2033+0+0 (crc 0 0 0) 0x3e03980 con 0x3740c00 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.473+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649062634s,-4.648939133s]) 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.473+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.473+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886958122s,-9.886825562s]) 2026-02-11T23:31:38.473+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 25 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 83.997924805s delta_ub -7.310290337s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x3740400 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.473+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310432434s,-7.310290337s]) 2026-02-11T23:31:38.473+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 25 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:38.473802+0000/91.308219910s send_stamp 81.421409607s delta_ub -9.886810303s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x3740800 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:38.473+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:38.473+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886962891s,-9.886810303s]) 2026-02-11T23:31:38.865+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:38.865+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:39.152+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:39.152+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:39.152+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:39.152+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.060 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:39.152+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:39.152, recovery is active?:false restrictions:<....> 2026-02-11T23:31:39.152+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:39.152+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:39.152+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:39.152+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:39.152+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:39.172+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:39.172+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:39.172+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:09.172216+0000) 2026-02-11T23:31:39.831+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:39.831+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:40.083+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 26 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:40.083703+0000/85.607742310s send_stamp 85.607742310s delta_ub -7.310290337s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b980 con 0x3f56800 2026-02-11T23:31:40.083+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310441017s,-7.310290337s]) 2026-02-11T23:31:40.083+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:40.083703+0000/85.607742310s send_stamp 92.918174744s delta_ub 7.310441017s) -- 0x567c480 con 0x3f56800 2026-02-11T23:31:40.083+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 26 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:40.083703+0000/85.607742310s send_stamp 85.607742310s delta_ub -7.310290337s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x3f56c00 2026-02-11T23:31:40.083+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:40.083+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:40.083+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:40.083+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:40.083+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310456753s,-7.310290337s]) 2026-02-11T23:31:40.083+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:40.083703+0000/85.607742310s send_stamp 92.918197632s delta_ub 7.310456753s) -- 0x567cf00 con 0x3f56c00 2026-02-11T23:31:40.083+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:40.083+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:40.083+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:40.083+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:40.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:40.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:40.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:10.172240+0000) 2026-02-11T23:31:40.171+0000 7fac616e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:40.174+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:40.174+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:40.174+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:40.174+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.056 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:40.174+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:40.175, recovery is active?:false restrictions:<....> 2026-02-11T23:31:40.174+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:40.174+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:40.174+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:40.174+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:40.174+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:40.553+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 28 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:40.553693+0000/91.407257080s send_stamp 91.407257080s delta_ub -1.980792522s) ==== 2033+0+0 (crc 0 0 0) 0x567c480 con 0x396a400 2026-02-11T23:31:40.553+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980899096s,-1.980792522s]) 2026-02-11T23:31:40.553+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 28 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:40.553693+0000/91.407257080s send_stamp 91.407257080s delta_ub -1.980792522s) ==== 2033+0+0 (crc 0 0 0) 0x567cf00 con 0x396a000 2026-02-11T23:31:40.553+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:40.553693+0000/91.407257080s send_stamp 93.388153076s delta_ub 1.980899096s) -- 0x3b1b980 con 0x396a400 2026-02-11T23:31:40.553+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:40.553+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:40.553+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:40.553+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:40.553+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980913758s,-1.980792522s]) 2026-02-11T23:31:40.553+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:40.553693+0000/91.407257080s send_stamp 93.388168335s delta_ub 1.980913758s) -- 0x3e02480 con 0x396a000 2026-02-11T23:31:40.553+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:40.553+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:40.553+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:40.553+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:40.880+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:40.880+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:41.172+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:41.172+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:41.172+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:11.172272+0000) 2026-02-11T23:31:41.221+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 24 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:41.221734+0000/89.407127380s send_stamp 89.407127380s delta_ub -4.648939133s) ==== 2033+0+0 (crc 0 0 0) 0x3e02480 con 0x3f56000 2026-02-11T23:31:41.221+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649076462s,-4.648939133s]) 2026-02-11T23:31:41.221+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 24 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:41.221734+0000/89.407127380s send_stamp 89.407127380s delta_ub -4.648939133s) ==== 2033+0+0 (crc 0 0 0) 0x3e03980 con 0x3f56400 2026-02-11T23:31:41.221+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:41.221734+0000/89.407127380s send_stamp 94.056205750s delta_ub 4.649076462s) -- 0x567cf00 con 0x3f56000 2026-02-11T23:31:41.221+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:41.221+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.221+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.221+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.221+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649090767s,-4.648939133s]) 2026-02-11T23:31:41.221+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:41.221734+0000/89.407127380s send_stamp 94.056221008s delta_ub 4.649090767s) -- 0x567c000 con 0x3f56400 2026-02-11T23:31:41.221+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:41.221+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.221+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.221+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.224+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:41.224+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:41.224+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:41.224+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:41.224+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:41.224+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.056 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:41.224+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:41.224, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:41.224+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:41.224+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:41.224+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:41.224+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:41.224+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:41.595+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1251 ==== pg_lease(1.0 pg_lease(ru 100.449119568s ub 108.449188232s int 16.000000000s) e118/118) ==== 58+0+0 (unknown 0 0 0) 0x567c000 con 0x3752000 2026-02-11T23:31:41.595+0000 7fac703fa640 15 osd.0 118 enqueue_peering_evt 1.0 epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 100.449119568s ub 108.449188232s int 16.000000000s) 2026-02-11T23:31:41.595+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 100.449119568s ub 108.449188232s int 16.000000000s)) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 100.449119568s ub 108.449188232s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 100.449119568s ub 108.449188232s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) pg 0x4a2a000 2026-02-11T23:31:41.595+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:41.595+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 100.449119568s ub 108.449188232s int 16.000000000s) 2026-02-11T23:31:41.595+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 100.449119568s ub 108.449188232s int 16.000000000s) 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -1.980792522s -> ru 102.429908752s 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until now 102.429908752s 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.980913758s -> ruub 110.430099487s 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 110.430099487s 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.595+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_lease_ack(1.0 pg_lease_ack(ruub 108.449188232s) e118/118) -- 0x5801a20 con 0x3752000 2026-02-11T23:31:41.595+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:41.595+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:41.897+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1259 ==== pg_lease(4.0 pg_lease(ru 92.845146179s ub 100.845268250s int 16.000000000s) e118/118) ==== 58+0+0 (unknown 0 0 0) 0x567c000 con 0x3e4a000 2026-02-11T23:31:41.897+0000 7fac703fa640 15 osd.0 118 enqueue_peering_evt 4.0 epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 92.845146179s ub 100.845268250s int 16.000000000s) 2026-02-11T23:31:41.897+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 92.845146179s ub 100.845268250s int 16.000000000s)) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 92.845146179s ub 100.845268250s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 92.845146179s ub 100.845268250s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) pg 0x4859000 2026-02-11T23:31:41.897+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:41.897+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 92.845146179s ub 100.845268250s int 16.000000000s) 2026-02-11T23:31:41.897+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 92.845146179s ub 100.845268250s int 16.000000000s) 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.886810303s -> ru 102.731956482s 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until now 102.731956482s 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.886962891s -> ruub 110.732231140s 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 110.732231140s 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.897+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_lease_ack(4.0 pg_lease_ack(ruub 100.845268250s) e118/118) -- 0x5613080 con 0x3e4a000 2026-02-11T23:31:41.897+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:41.897+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:41.905+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:41.905+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:41.960+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 27 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:41.960339+0000/84.907844543s send_stamp 84.907844543s delta_ub -9.886810303s) ==== 2033+0+0 (crc 0 0 0) 0x567c000 con 0x3f57400 2026-02-11T23:31:41.960+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886952400s,-9.886810303s]) 2026-02-11T23:31:41.960+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:41.960339+0000/84.907844543s send_stamp 94.794799805s delta_ub 9.886952400s) -- 0x3e03980 con 0x3f57400 2026-02-11T23:31:41.960+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:41.960+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.960+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.960+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.960+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 27 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:41.960339+0000/84.907844543s send_stamp 84.907844543s delta_ub -9.886810303s) ==== 2033+0+0 (crc 0 0 0) 0x3b1b980 con 0x3f57800 2026-02-11T23:31:41.960+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886963844s,-9.886810303s]) 2026-02-11T23:31:41.960+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:41.960339+0000/84.907844543s send_stamp 94.794807434s delta_ub 9.886963844s) -- 0x567c480 con 0x3f57800 2026-02-11T23:31:41.960+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:41.960+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.960+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.960+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.973+0000 7fac4d160640 5 osd.0 118 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x537bf3/0x5de000, compress 0x0/0x0/0x0, omap 0xc287, meta 0x1a43d79), peers [1,2,3,4] op hist []) 2026-02-11T23:31:41.973+0000 7fac4d160640 20 osd.0 118 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:41.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 94.808357239s delta_ub 1.980913758s) -- 0x3e03b00 con 0x396b400 2026-02-11T23:31:41.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 94.808357239s delta_ub 1.980913758s) -- 0x3e03500 con 0x3752400 2026-02-11T23:31:41.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 94.808357239s delta_ub 4.649090767s) -- 0x3e02f00 con 0x396b800 2026-02-11T23:31:41.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 94.808357239s delta_ub 4.649090767s) -- 0x3e02d80 con 0x34c1000 2026-02-11T23:31:41.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 94.808357239s delta_ub 7.310456753s) -- 0x3e02900 con 0x3740000 2026-02-11T23:31:41.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 94.808357239s delta_ub 7.310456753s) -- 0x3e02300 con 0x3740400 2026-02-11T23:31:41.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 94.808357239s delta_ub 9.886963844s) -- 0x3e02180 con 0x3740800 2026-02-11T23:31:41.973+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 94.808357239s delta_ub 9.886963844s) -- 0x3e03380 con 0x3740c00 2026-02-11T23:31:41.973+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 26 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 87.498039246s delta_ub -7.310317993s) ==== 2033+0+0 (crc 0 0 0) 0x3e03380 con 0x3740000 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.973+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 29 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 92.827560425s delta_ub -1.980798006s) ==== 2033+0+0 (crc 0 0 0) 0x3e02d80 con 0x3752400 2026-02-11T23:31:41.973+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 28 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 90.159400940s delta_ub -4.648961067s) ==== 2033+0+0 (crc 0 0 0) 0x3e02180 con 0x396b800 2026-02-11T23:31:41.973+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310422421s,-7.310317993s]) 2026-02-11T23:31:41.973+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:41.973+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.973+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.973+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.973+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980910778s,-1.980798006s]) 2026-02-11T23:31:41.973+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 29 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 92.827575684s delta_ub -1.980785251s) ==== 2033+0+0 (crc 0 0 0) 0x3e03380 con 0x396b400 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.973+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980904341s,-1.980785251s]) 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.973+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.973+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 28 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 90.159423828s delta_ub -4.648936272s) ==== 2033+0+0 (crc 0 0 0) 0x3e02d80 con 0x34c1000 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649087429s,-4.648961067s]) 2026-02-11T23:31:41.973+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 26 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 84.921554565s delta_ub -9.886797905s) ==== 2033+0+0 (crc 0 0 0) 0x3e03380 con 0x3740c00 2026-02-11T23:31:41.973+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:41.973+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.973+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.973+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.973+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 26 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 87.498046875s delta_ub -7.310305595s) ==== 2033+0+0 (crc 0 0 0) 0x3e02180 con 0x3740400 2026-02-11T23:31:41.973+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649075031s,-4.648936272s]) 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.973+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.973+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886947632s,-9.886797905s]) 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.973+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310461521s,-7.310305595s]) 2026-02-11T23:31:41.973+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 26 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:41.973940+0000/94.808357239s send_stamp 84.921531677s delta_ub -9.886825562s) ==== 2033+0+0 (crc 0 0 0) 0x3e02180 con 0x3740800 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:41.973+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:41.973+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886991501s,-9.886825562s]) 2026-02-11T23:31:42.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:42.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:42.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:12.172297+0000) 2026-02-11T23:31:42.233+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:42.233+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:42.233+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:42.233+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:42.233+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:42.233+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.056 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:42.233+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:42.234, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:42.233+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:42.233+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:42.233+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:42.233+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:42.233+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:42.253+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 29 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:42.253824+0000/93.107391357s send_stamp 93.107391357s delta_ub -1.980785251s) ==== 2033+0+0 (crc 0 0 0) 0x3e02d80 con 0x396a400 2026-02-11T23:31:42.253+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980904460s,-1.980785251s]) 2026-02-11T23:31:42.253+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 29 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:42.253824+0000/93.107391357s send_stamp 93.107391357s delta_ub -1.980785251s) ==== 2033+0+0 (crc 0 0 0) 0x3e02180 con 0x396a000 2026-02-11T23:31:42.253+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:42.253824+0000/93.107391357s send_stamp 95.088294983s delta_ub 1.980904460s) -- 0x3e03500 con 0x396a400 2026-02-11T23:31:42.253+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:42.253+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.253+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.253+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.253+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980933189s,-1.980785251s]) 2026-02-11T23:31:42.253+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:42.253824+0000/93.107391357s send_stamp 95.088317871s delta_ub 1.980933189s) -- 0x3e02300 con 0x396a000 2026-02-11T23:31:42.253+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:42.253+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.253+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.253+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.474+0000 7fac4d160640 5 osd.0 118 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x537bf3/0x5de000, compress 0x0/0x0/0x0, omap 0xc287, meta 0x1a43d79), peers [1,2,3,4] op hist []) 2026-02-11T23:31:42.474+0000 7fac4d160640 20 osd.0 118 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:42.474+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 95.308486938s delta_ub 1.980933189s) -- 0x3e03680 con 0x396b400 2026-02-11T23:31:42.474+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 95.308486938s delta_ub 1.980933189s) -- 0x3e02a80 con 0x3752400 2026-02-11T23:31:42.474+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 95.308486938s delta_ub 4.649075031s) -- 0x3e02780 con 0x396b800 2026-02-11T23:31:42.474+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 95.308486938s delta_ub 4.649075031s) -- 0x3e02000 con 0x34c1000 2026-02-11T23:31:42.474+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 95.308486938s delta_ub 7.310461521s) -- 0x3e03080 con 0x3740000 2026-02-11T23:31:42.474+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 95.308486938s delta_ub 7.310461521s) -- 0x3919980 con 0x3740400 2026-02-11T23:31:42.474+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 95.308486938s delta_ub 9.886991501s) -- 0x567ca80 con 0x3740800 2026-02-11T23:31:42.474+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 95.308486938s delta_ub 9.886991501s) -- 0x567d080 con 0x3740c00 2026-02-11T23:31:42.474+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 30 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 93.327682495s delta_ub -1.980809689s) ==== 2033+0+0 (crc 0 0 0) 0x567d080 con 0x396b400 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.474+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 30 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 93.327697754s delta_ub -1.980796456s) ==== 2033+0+0 (crc 0 0 0) 0x3e02000 con 0x3752400 2026-02-11T23:31:42.474+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.474+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 29 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 90.659523010s delta_ub -4.648965836s) ==== 2033+0+0 (crc 0 0 0) 0x567ca80 con 0x396b800 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980904460s,-1.980809689s]) 2026-02-11T23:31:42.474+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 27 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 87.998176575s delta_ub -7.310315132s) ==== 2033+0+0 (crc 0 0 0) 0x567d080 con 0x3740000 2026-02-11T23:31:42.474+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:42.474+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.474+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.474+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.474+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980900168s,-1.980796456s]) 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.474+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.474+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 29 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 90.659538269s delta_ub -4.648950100s) ==== 2033+0+0 (crc 0 0 0) 0x3e02000 con 0x34c1000 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649083614s,-4.648965836s]) 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.474+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310444355s,-7.310315132s]) 2026-02-11T23:31:42.474+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 27 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 87.998184204s delta_ub -7.310304642s) ==== 2033+0+0 (crc 0 0 0) 0x567ca80 con 0x3740400 2026-02-11T23:31:42.474+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:42.474+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.474+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.474+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.474+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649085999s,-4.648950100s]) 2026-02-11T23:31:42.474+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 27 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 85.421684265s delta_ub -9.886807442s) ==== 2033+0+0 (crc 0 0 0) 0x567d080 con 0x3740c00 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.474+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.474+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886947632s,-9.886807442s]) 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.474+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310452938s,-7.310304642s]) 2026-02-11T23:31:42.474+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 27 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:42.474074+0000/95.308486938s send_stamp 85.421669006s delta_ub -9.886820793s) ==== 2033+0+0 (crc 0 0 0) 0x567ca80 con 0x3740800 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.474+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.474+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886980057s,-9.886820793s]) 2026-02-11T23:31:42.753+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 30 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:42.753982+0000/93.607551575s send_stamp 93.607551575s delta_ub -1.980796456s) ==== 2033+0+0 (crc 0 0 0) 0x3e02000 con 0x396a400 2026-02-11T23:31:42.753+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980901241s,-1.980796456s]) 2026-02-11T23:31:42.753+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 30 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:42.753982+0000/93.607551575s send_stamp 93.607551575s delta_ub -1.980796456s) ==== 2033+0+0 (crc 0 0 0) 0x567ca80 con 0x396a000 2026-02-11T23:31:42.753+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:42.753982+0000/93.607551575s send_stamp 95.588447571s delta_ub 1.980901241s) -- 0x3e02a80 con 0x396a400 2026-02-11T23:31:42.753+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:42.753+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.753+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.753+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.753+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980918527s,-1.980796456s]) 2026-02-11T23:31:42.753+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:42.753982+0000/93.607551575s send_stamp 95.588470459s delta_ub 1.980918527s) -- 0x3919980 con 0x396a000 2026-02-11T23:31:42.753+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:42.753+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.753+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.753+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.931+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:42.931+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:42.974+0000 7fac4d160640 5 osd.0 118 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x537bf3/0x5de000, compress 0x0/0x0/0x0, omap 0xc287, meta 0x1a43d79), peers [1,2,3,4] op hist []) 2026-02-11T23:31:42.974+0000 7fac4d160640 20 osd.0 118 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:42.974+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 95.808631897s delta_ub 1.980918527s) -- 0x567d680 con 0x396b400 2026-02-11T23:31:42.974+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 95.808631897s delta_ub 1.980918527s) -- 0x567d500 con 0x3752400 2026-02-11T23:31:42.974+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 95.808631897s delta_ub 4.649085999s) -- 0x567d800 con 0x396b800 2026-02-11T23:31:42.974+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 95.808631897s delta_ub 4.649085999s) -- 0x568e000 con 0x34c1000 2026-02-11T23:31:42.974+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 95.808631897s delta_ub 7.310452938s) -- 0x568e180 con 0x3740000 2026-02-11T23:31:42.974+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 95.808631897s delta_ub 7.310452938s) -- 0x568e300 con 0x3740400 2026-02-11T23:31:42.974+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 95.808631897s delta_ub 9.886980057s) -- 0x568e480 con 0x3740800 2026-02-11T23:31:42.974+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 95.808631897s delta_ub 9.886980057s) -- 0x568e600 con 0x3740c00 2026-02-11T23:31:42.974+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 30 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 91.159675598s delta_ub -4.648958683s) ==== 2033+0+0 (crc 0 0 0) 0x568e480 con 0x396b800 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:42.974+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 31 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 93.827850342s delta_ub -1.980780840s) ==== 2033+0+0 (crc 0 0 0) 0x568e000 con 0x3752400 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.974+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 31 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 93.827842712s delta_ub -1.980796218s) ==== 2033+0+0 (crc 0 0 0) 0x568e600 con 0x396b400 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.974+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649073601s,-4.648958683s]) 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.974+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980922103s,-1.980796218s]) 2026-02-11T23:31:42.974+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 28 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 88.498359680s delta_ub -7.310279846s) ==== 2033+0+0 (crc 0 0 0) 0x568e480 con 0x3740400 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.974+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310411930s,-7.310279846s]) 2026-02-11T23:31:42.974+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:42.974+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.974+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.974+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.974+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 28 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 85.921844482s delta_ub -9.886795998s) ==== 2033+0+0 (crc 0 0 0) 0x568e480 con 0x3740800 2026-02-11T23:31:42.974+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980916023s,-1.980780840s]) 2026-02-11T23:31:42.974+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 28 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 88.498336792s delta_ub -7.310295582s) ==== 2033+0+0 (crc 0 0 0) 0x568e600 con 0x3740000 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.974+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310439110s,-7.310295582s]) 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.974+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.974+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886947632s,-9.886795998s]) 2026-02-11T23:31:42.974+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 30 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 91.159690857s delta_ub -4.648940563s) ==== 2033+0+0 (crc 0 0 0) 0x568e000 con 0x34c1000 2026-02-11T23:31:42.974+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:42.974+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.974+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 28 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:42.974219+0000/95.808631897s send_stamp 85.921852112s delta_ub -9.886785507s) ==== 2033+0+0 (crc 0 0 0) 0x568e600 con 0x3740c00 2026-02-11T23:31:42.974+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.974+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.974+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649097443s,-4.648940563s]) 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:42.974+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:42.974+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886951447s,-9.886785507s]) 2026-02-11T23:31:43.015+0000 7fac60ee5640 20 osd.0 118 reports for 0 queries 2026-02-11T23:31:43.015+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4ff0a80 con 0x396a800 2026-02-11T23:31:43.015+0000 7fac60ee5640 15 osd.0 118 collect_pg_stats 2026-02-11T23:31:43.015+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836504 v 0) -- 0x567e900 con 0x396a800 2026-02-11T23:31:43.172+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:43.172+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:43.172+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:13.172323+0000) 2026-02-11T23:31:43.241+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:43.241+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:43.241+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:43.241+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:43.241+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:43.241+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.056 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:43.241+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:43.241, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:43.241+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:43.241+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:43.241+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:43.241+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:43.241+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:43.896+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:43.896+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:44.170+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:44.170+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:44.170+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:14.172349+0000) 2026-02-11T23:31:44.182+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 27 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:44.183865+0000/89.707908630s send_stamp 89.707908630s delta_ub -7.310279846s) ==== 2033+0+0 (crc 0 0 0) 0x568e480 con 0x3f56c00 2026-02-11T23:31:44.182+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310440540s,-7.310279846s]) 2026-02-11T23:31:44.182+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 27 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:44.183865+0000/89.707908630s send_stamp 89.707908630s delta_ub -7.310279846s) ==== 2033+0+0 (crc 0 0 0) 0x568e000 con 0x3f56800 2026-02-11T23:31:44.182+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:44.183865+0000/89.707908630s send_stamp 97.018341064s delta_ub 7.310440540s) -- 0x568e300 con 0x3f56c00 2026-02-11T23:31:44.182+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:44.182+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:44.182+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:44.182+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:44.182+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310457706s,-7.310279846s]) 2026-02-11T23:31:44.182+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:44.183865+0000/89.707908630s send_stamp 97.018363953s delta_ub 7.310457706s) -- 0x567d500 con 0x3f56800 2026-02-11T23:31:44.182+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:44.182+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:44.182+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:44.182+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:44.247+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:44.247+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:44.247+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:44.247+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:44.247+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:44.247+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.056 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:44.247+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:44.248, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:44.247+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:44.247+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:44.247+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:44.247+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:44.247+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:44.259+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 28 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:44.260500+0000/87.208007812s send_stamp 87.208007812s delta_ub -9.886785507s) ==== 2033+0+0 (crc 0 0 0) 0x567d500 con 0x3f57800 2026-02-11T23:31:44.259+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886962891s,-9.886785507s]) 2026-02-11T23:31:44.259+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:44.260500+0000/87.208007812s send_stamp 97.094970703s delta_ub 9.886962891s) -- 0x568e000 con 0x3f57800 2026-02-11T23:31:44.259+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:44.259+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:44.259+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:44.259+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:44.259+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 28 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:44.260500+0000/87.208007812s send_stamp 87.208007812s delta_ub -9.886785507s) ==== 2033+0+0 (crc 0 0 0) 0x568e600 con 0x3f57400 2026-02-11T23:31:44.259+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886981010s,-9.886785507s]) 2026-02-11T23:31:44.259+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:44.260500+0000/87.208007812s send_stamp 97.094993591s delta_ub 9.886981010s) -- 0x568e180 con 0x3f57400 2026-02-11T23:31:44.259+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:44.259+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:44.259+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:44.259+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:44.889+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:44.889+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:45.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:45.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:45.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:15.172375+0000) 2026-02-11T23:31:45.214+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:45.214+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:45.214+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:45.214+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:45.214+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:45.214+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.051 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:45.214+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:45.215, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:45.214+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:45.214+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:45.214+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:45.214+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:45.214+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:45.320+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 25 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:45.321869+0000/93.507263184s send_stamp 93.507263184s delta_ub -4.648940563s) ==== 2033+0+0 (crc 0 0 0) 0x568e180 con 0x3f56400 2026-02-11T23:31:45.320+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649064541s,-4.648940563s]) 2026-02-11T23:31:45.320+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 25 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:45.321869+0000/93.507263184s send_stamp 93.507263184s delta_ub -4.648940563s) ==== 2033+0+0 (crc 0 0 0) 0x568e300 con 0x3f56000 2026-02-11T23:31:45.320+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:45.321869+0000/93.507263184s send_stamp 98.156333923s delta_ub 4.649064541s) -- 0x568e600 con 0x3f56400 2026-02-11T23:31:45.320+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:45.320+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:45.320+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:45.320+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:45.320+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649075031s,-4.648940563s]) 2026-02-11T23:31:45.320+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:45.321869+0000/93.507263184s send_stamp 98.156341553s delta_ub 4.649075031s) -- 0x568e480 con 0x3f56000 2026-02-11T23:31:45.320+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:45.320+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:45.320+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:45.320+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:45.653+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 31 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:45.654119+0000/96.507682800s send_stamp 96.507682800s delta_ub -1.980780840s) ==== 2033+0+0 (crc 0 0 0) 0x568e000 con 0x396a400 2026-02-11T23:31:45.653+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 31 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:45.654119+0000/96.507682800s send_stamp 96.507682800s delta_ub -1.980780840s) ==== 2033+0+0 (crc 0 0 0) 0x568e480 con 0x396a000 2026-02-11T23:31:45.653+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980896711s,-1.980780840s]) 2026-02-11T23:31:45.653+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:45.654119+0000/96.507682800s send_stamp 98.488578796s delta_ub 1.980896711s) -- 0x567d500 con 0x396a400 2026-02-11T23:31:45.653+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:45.653+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:45.653+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:45.653+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:45.653+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980908394s,-1.980780840s]) 2026-02-11T23:31:45.653+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:45.654119+0000/96.507682800s send_stamp 98.488594055s delta_ub 1.980908394s) -- 0x568e300 con 0x396a000 2026-02-11T23:31:45.653+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:45.653+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:45.653+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:45.653+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:45.901+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:45.901+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:46.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:46.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:46.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:16.172401+0000) 2026-02-11T23:31:46.257+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:46.257+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:46.257+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:46.257+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.051 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:46.257+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:46.258, recovery is active?:false restrictions:<....> 2026-02-11T23:31:46.257+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:46.257+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:46.257+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:46.257+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:46.257+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:46.473+0000 7fac4d160640 5 osd.0 118 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x537bf3/0x5de000, compress 0x0/0x0/0x0, omap 0xc287, meta 0x1a43d79), peers [1,2,3,4] op hist []) 2026-02-11T23:31:46.473+0000 7fac4d160640 20 osd.0 118 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:46.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 99.308792114s delta_ub 1.980908394s) -- 0x568e780 con 0x396b400 2026-02-11T23:31:46.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 99.308792114s delta_ub 1.980908394s) -- 0x568e900 con 0x3752400 2026-02-11T23:31:46.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 99.308792114s delta_ub 4.649075031s) -- 0x568ea80 con 0x396b800 2026-02-11T23:31:46.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 99.308792114s delta_ub 4.649075031s) -- 0x568ec00 con 0x34c1000 2026-02-11T23:31:46.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 99.308792114s delta_ub 7.310457706s) -- 0x568ed80 con 0x3740000 2026-02-11T23:31:46.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 99.308792114s delta_ub 7.310457706s) -- 0x568ef00 con 0x3740400 2026-02-11T23:31:46.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 99.308792114s delta_ub 9.886981010s) -- 0x568f080 con 0x3740800 2026-02-11T23:31:46.473+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 99.308792114s delta_ub 9.886981010s) -- 0x568f200 con 0x3740c00 2026-02-11T23:31:46.473+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 32 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 97.328002930s delta_ub -1.980785608s) ==== 2033+0+0 (crc 0 0 0) 0x568f200 con 0x396b400 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:46.473+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980892420s,-1.980785608s]) 2026-02-11T23:31:46.473+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 29 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 91.998497009s delta_ub -7.310296535s) ==== 2033+0+0 (crc 0 0 0) 0x568f200 con 0x3740000 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:46.473+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 31 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 94.659851074s delta_ub -4.648940086s) ==== 2033+0+0 (crc 0 0 0) 0x568f080 con 0x396b800 2026-02-11T23:31:46.473+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310419559s,-7.310296535s]) 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:46.473+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:46.473+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 31 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 94.659873962s delta_ub -4.648923874s) ==== 2033+0+0 (crc 0 0 0) 0x568ec00 con 0x34c1000 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649071217s,-4.648940086s]) 2026-02-11T23:31:46.473+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 29 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 89.422004700s delta_ub -9.886794090s) ==== 2033+0+0 (crc 0 0 0) 0x568f200 con 0x3740c00 2026-02-11T23:31:46.473+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:46.473+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:46.473+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:46.473+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:46.473+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649070263s,-4.648923874s]) 2026-02-11T23:31:46.473+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 29 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 91.998512268s delta_ub -7.310282230s) ==== 2033+0+0 (crc 0 0 0) 0x568f080 con 0x3740400 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:46.473+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310437202s,-7.310282230s]) 2026-02-11T23:31:46.473+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 32 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 97.328041077s delta_ub -1.980751991s) ==== 2033+0+0 (crc 0 0 0) 0x568ec00 con 0x3752400 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:46.473+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:46.473+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886957169s,-9.886794090s]) 2026-02-11T23:31:46.473+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 29 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:46.474378+0000/99.308792114s send_stamp 89.421981812s delta_ub -9.886810303s) ==== 2033+0+0 (crc 0 0 0) 0x568f080 con 0x3740800 2026-02-11T23:31:46.473+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:46.473+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:46.473+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:46.473+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:46.473+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980922937s,-1.980751991s]) 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:46.473+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:46.473+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886989594s,-9.886810303s]) 2026-02-11T23:31:46.865+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:46.865+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:47.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:47.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:47.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:17.172427+0000) 2026-02-11T23:31:47.236+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:47.236+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:47.236+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:47.236+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.051 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:47.236+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:47.237, recovery is active?:false restrictions:<....> 2026-02-11T23:31:47.236+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:47.236+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:47.236+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:47.236+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:47.236+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:47.836+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:47.836+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:48.014+0000 7fac60ee5640 20 osd.0 118 reports for 0 queries 2026-02-11T23:31:48.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4ff0e00 con 0x396a800 2026-02-11T23:31:48.014+0000 7fac60ee5640 15 osd.0 118 collect_pg_stats 2026-02-11T23:31:48.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836505 v 0) -- 0x567ec00 con 0x396a800 2026-02-11T23:31:48.106+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> conn(0x48e7000 0x40ec100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:31:48.106+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> conn(0x48e7000 0x40ec100 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:31:48.106+0000 7fac71bfd640 10 osd.0 118 ms_handle_fast_authentication new session 0x5678380 con 0x48e7000 entity client.admin addr 2026-02-11T23:31:48.106+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/921035377 conn(0x48e7000 0x40ec100 crc :-1 s=READY pgs=3 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=client.7931 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:31:48.106+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.7931 127.0.0.1:0/921035377 1 ==== command(tid 1: {"prefix": "get_command_descriptions"}) ==== 62+0+0 (crc 0 0 0) 0x25d94a0 con 0x48e7000 2026-02-11T23:31:48.106+0000 7fac626e8640 20 osd.0 118 OSD::ms_dispatch: command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:31:48.106+0000 7fac626e8640 20 osd.0 118 _dispatch 0x25d94a0 command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:31:48.107+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/921035377 -- command_reply(tid 1: 0 ) -- 0x25d9340 con 0x48e7000 2026-02-11T23:31:48.114+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.7931 127.0.0.1:0/921035377 2 ==== command(tid 2: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x25d9340 con 0x48e7000 2026-02-11T23:31:48.114+0000 7fac626e8640 20 osd.0 118 OSD::ms_dispatch: command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:48.114+0000 7fac626e8640 20 osd.0 118 _dispatch 0x25d9340 command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:48.114+0000 7fac713fc640 15 osd.0 118 collect_pg_stats 2026-02-11T23:31:48.114+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836506 v 0) -- 0x3df0300 con 0x396a800 2026-02-11T23:31:48.114+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/921035377 -- command_reply(tid 2: 0 ) -- 0x25d94a0 con 0x48e7000 2026-02-11T23:31:48.120+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.7931 127.0.0.1:0/921035377 3 ==== command(tid 3: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x25d94a0 con 0x48e7000 2026-02-11T23:31:48.120+0000 7fac626e8640 20 osd.0 118 OSD::ms_dispatch: command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:48.120+0000 7fac626e8640 20 osd.0 118 _dispatch 0x25d94a0 command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:48.120+0000 7fac713fc640 15 osd.0 118 collect_pg_stats 2026-02-11T23:31:48.120+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836507 v 0) -- 0x3df0600 con 0x396a800 2026-02-11T23:31:48.120+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/921035377 -- command_reply(tid 3: 0 ) -- 0x25d9340 con 0x48e7000 2026-02-11T23:31:48.120+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/921035377 conn(0x48e7000 msgr2=0x40ec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 79 2026-02-11T23:31:48.120+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/921035377 conn(0x48e7000 msgr2=0x40ec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-11T23:31:48.120+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/921035377 conn(0x48e7000 0x40ec100 crc :-1 s=READY pgs=3 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-11T23:31:48.120+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/921035377 conn(0x48e7000 0x40ec100 crc :-1 s=READY pgs=3 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:31:48.120+0000 7fac626e8640 2 osd.0 118 ms_handle_reset con 0x48e7000 session 0x5678380 2026-02-11T23:31:48.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:48.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:48.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:18.172498+0000) 2026-02-11T23:31:48.273+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:48.273+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:48.273+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:48.273+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.051 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:48.273+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:48.274, recovery is active?:false restrictions:<....> 2026-02-11T23:31:48.273+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:48.273+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:48.273+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:48.273+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:48.273+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:48.860+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:48.860+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:48.883+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 28 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:48.884039+0000/94.408073425s send_stamp 94.408073425s delta_ub -7.310282230s) ==== 2033+0+0 (crc 0 0 0) 0x568f080 con 0x3f56c00 2026-02-11T23:31:48.883+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 28 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:48.884039+0000/94.408073425s send_stamp 94.408073425s delta_ub -7.310282230s) ==== 2033+0+0 (crc 0 0 0) 0x3da2600 con 0x3f56800 2026-02-11T23:31:48.883+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310482502s,-7.310282230s]) 2026-02-11T23:31:48.883+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:48.884039+0000/94.408073425s send_stamp 101.718559265s delta_ub 7.310482502s) -- 0x568ef00 con 0x3f56c00 2026-02-11T23:31:48.883+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:48.883+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:48.883+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:48.883+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:48.883+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310510635s,-7.310282230s]) 2026-02-11T23:31:48.883+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:48.884039+0000/94.408073425s send_stamp 101.718589783s delta_ub 7.310510635s) -- 0x3da3980 con 0x3f56800 2026-02-11T23:31:48.883+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:48.883+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:48.883+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:48.883+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:49.153+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 32 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:49.154282+0000/100.007843018s send_stamp 100.007843018s delta_ub -1.980751991s) ==== 2033+0+0 (crc 0 0 0) 0x3da3980 con 0x396a400 2026-02-11T23:31:49.153+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980931044s,-1.980751991s]) 2026-02-11T23:31:49.153+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:49.154282+0000/100.007843018s send_stamp 101.988777161s delta_ub 1.980931044s) -- 0x3da2600 con 0x396a400 2026-02-11T23:31:49.153+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 32 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:49.154282+0000/100.007843018s send_stamp 100.007843018s delta_ub -1.980751991s) ==== 2033+0+0 (crc 0 0 0) 0x568ef00 con 0x396a000 2026-02-11T23:31:49.153+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:49.153+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:49.153+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:49.153+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:49.153+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980950236s,-1.980751991s]) 2026-02-11T23:31:49.153+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:49.154282+0000/100.007843018s send_stamp 101.988792419s delta_ub 1.980950236s) -- 0x568f080 con 0x396a000 2026-02-11T23:31:49.153+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:49.153+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:49.153+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:49.153+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:49.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:49.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:49.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:19.172541+0000) 2026-02-11T23:31:49.267+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:49.267+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:49.267+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:49.267+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:49.267+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:49.267+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.051 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:49.267+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:49.268, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:49.267+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:49.267+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:49.267+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:49.267+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:49.267+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:49.383+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 29 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:49.384212+0000/94.908248901s send_stamp 94.908248901s delta_ub -7.310298920s) ==== 2033+0+0 (crc 0 0 0) 0x3da2600 con 0x3f56800 2026-02-11T23:31:49.383+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310437202s,-7.310298920s]) 2026-02-11T23:31:49.383+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 29 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:49.384212+0000/94.908248901s send_stamp 94.908248901s delta_ub -7.310298920s) ==== 2033+0+0 (crc 0 0 0) 0x568f080 con 0x3f56c00 2026-02-11T23:31:49.383+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:49.384212+0000/94.908248901s send_stamp 102.218688965s delta_ub 7.310437202s) -- 0x3da3980 con 0x3f56800 2026-02-11T23:31:49.383+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:49.383+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:49.383+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:49.383+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:49.383+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310453415s,-7.310298920s]) 2026-02-11T23:31:49.383+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:49.384212+0000/94.908248901s send_stamp 102.218704224s delta_ub 7.310453415s) -- 0x568ef00 con 0x3f56c00 2026-02-11T23:31:49.383+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:49.383+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:49.383+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:49.383+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:49.559+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 29 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:49.560671+0000/92.508178711s send_stamp 92.508178711s delta_ub -9.886794090s) ==== 2033+0+0 (crc 0 0 0) 0x568f200 con 0x3f57400 2026-02-11T23:31:49.559+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 29 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:49.560671+0000/92.508178711s send_stamp 92.508178711s delta_ub -9.886794090s) ==== 2033+0+0 (crc 0 0 0) 0x3da3980 con 0x3f57800 2026-02-11T23:31:49.559+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886974335s,-9.886794090s]) 2026-02-11T23:31:49.559+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:49.560671+0000/92.508178711s send_stamp 102.395149231s delta_ub 9.886974335s) -- 0x568ed80 con 0x3f57400 2026-02-11T23:31:49.559+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:49.559+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:49.559+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:49.559+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:49.559+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886998177s,-9.886794090s]) 2026-02-11T23:31:49.559+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:49.560671+0000/92.508178711s send_stamp 102.395172119s delta_ub 9.886998177s) -- 0x3da2600 con 0x3f57800 2026-02-11T23:31:49.559+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:49.559+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:49.559+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:49.559+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:49.594+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.1 v2:127.0.0.1:6812/4244365097 1252 ==== pg_lease(1.0 pg_lease(ru 108.449188232s ub 116.449356079s int 16.000000000s) e118/118) ==== 58+0+0 (unknown 0 0 0) 0x568ed80 con 0x3752000 2026-02-11T23:31:49.594+0000 7fac703fa640 15 osd.0 118 enqueue_peering_evt 1.0 epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 108.449188232s ub 116.449356079s int 16.000000000s) 2026-02-11T23:31:49.594+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 108.449188232s ub 116.449356079s int 16.000000000s)) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 108.449188232s ub 116.449356079s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 op_wq(0) _process 1.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 108.449188232s ub 116.449356079s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) pg 0x4a2a000 2026-02-11T23:31:49.594+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:49.594+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.1 pg_lease(ru 108.449188232s ub 116.449356079s int 16.000000000s) 2026-02-11T23:31:49.594+0000 7fac51168640 10 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 108.449188232s ub 116.449356079s int 16.000000000s) 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -1.980751991s -> ru 110.429939270s 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until now 110.429939270s 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -1.980950236s -> ruub 118.430305481s 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 pg_epoch: 118 pg[1.0( v 107'600 (107'590,107'600] local-lis/les=114/115 n=600 ec=24/24 lis/c=114/114 les/c/f=115/115/118 sis=114) [1,0] r=1 lpr=114 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 118.430305481s 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:49.594+0000 7fac51168640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6812/4244365097,v1:127.0.0.1:6813/4244365097] -- pg_lease_ack(1.0 pg_lease_ack(ruub 116.449356079s) e118/118) -- 0x56129a0 con 0x3752000 2026-02-11T23:31:49.594+0000 7fac51168640 10 log is not dirty 2026-02-11T23:31:49.594+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:49.653+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 33 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:49.654478+0000/100.508041382s send_stamp 100.508041382s delta_ub -1.980789781s) ==== 2033+0+0 (crc 0 0 0) 0x3da2600 con 0x396a400 2026-02-11T23:31:49.653+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980890512s,-1.980789781s]) 2026-02-11T23:31:49.653+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:49.654478+0000/100.508041382s send_stamp 102.488929749s delta_ub 1.980890512s) -- 0x3da3980 con 0x396a400 2026-02-11T23:31:49.653+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:49.653+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:49.653+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:49.653+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:49.653+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 33 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:49.654478+0000/100.508041382s send_stamp 100.508041382s delta_ub -1.980789781s) ==== 2033+0+0 (crc 0 0 0) 0x568ef00 con 0x396a000 2026-02-11T23:31:49.653+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980901718s,-1.980789781s]) 2026-02-11T23:31:49.653+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:49.654478+0000/100.508041382s send_stamp 102.488945007s delta_ub 1.980901718s) -- 0x568f080 con 0x396a000 2026-02-11T23:31:49.653+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:49.653+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:49.653+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:49.653+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:49.826+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:49.826+0000 7fac68beb640 10 osd.0 118 maybe_update_heartbeat_peers forcing update after 20.773723 seconds 2026-02-11T23:31:49.826+0000 7fac68beb640 10 osd.0 118 reset_heartbeat_peers 2026-02-11T23:31:49.826+0000 7fac68beb640 10 osd.0 118 maybe_update_heartbeat_peers updating 2026-02-11T23:31:49.826+0000 7fac68beb640 10 osd.0 118 adding neighbor peer osd.1 2026-02-11T23:31:49.826+0000 7fac68beb640 10 osd.0 118 adding neighbor peer osd.4 2026-02-11T23:31:49.826+0000 7fac68beb640 10 osd.0 118 maybe_update_heartbeat_peers 4 peers, extras 1,2,3,4 2026-02-11T23:31:49.826+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:49.896+0000 7fac703fa640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] <== osd.4 v2:127.0.0.1:6836/864727417 1260 ==== pg_lease(4.0 pg_lease(ru 100.845268250s ub 108.845390320s int 16.000000000s) e118/118) ==== 58+0+0 (unknown 0 0 0) 0x568ed80 con 0x3e4a000 2026-02-11T23:31:49.896+0000 7fac703fa640 15 osd.0 118 enqueue_peering_evt 4.0 epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 100.845268250s ub 108.845390320s int 16.000000000s) 2026-02-11T23:31:49.896+0000 7fac703fa640 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 100.845268250s ub 108.845390320s int 16.000000000s)) class_id 2 prio 255 cost 10 e118) 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process <> waiting <> waiting_peering {} 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 100.845268250s ub 108.845390320s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) queued 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 op_wq(0) _process 4.0 to_process waiting <> waiting_peering {} 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 op_wq(0) _process OpSchedulerItem(4.0 PGPeeringEvent(epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 100.845268250s ub 108.845390320s int 16.000000000s)) class_id 2immediate prio 255 cost 10 e118) pg 0x4859000 2026-02-11T23:31:49.896+0000 7fac51168640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:49.896+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] do_peering_event: epoch_sent: 118 epoch_requested: 118 MLease epoch 118 from osd.4 pg_lease(ru 100.845268250s ub 108.845390320s int 16.000000000s) 2026-02-11T23:31:49.896+0000 7fac55170640 10 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease pg_lease(ru 100.845268250s ub 108.845390320s int 16.000000000s) 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_ub -9.886794090s -> ru 110.732055664s 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until now 110.732055664s 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease peer_clock_delta_lb -9.886998177s -> ruub 118.732391357s 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 pg_epoch: 118 pg[4.0( v 107'600 (107'590,107'600] local-lis/les=116/117 n=600 ec=45/45 lis/c=116/116 les/c/f=117/117/118 sis=116) [4,0] r=1 lpr=116 crt=107'600 active mbc={}] PeeringState::proc_lease readable_until_ub now 118.732391357s 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:49.896+0000 7fac55170640 1 -- [v2:127.0.0.1:6802/460666604,v1:127.0.0.1:6803/460666604] --> [v2:127.0.0.1:6836/864727417,v1:127.0.0.1:6837/864727417] -- pg_lease_ack(4.0 pg_lease_ack(ruub 108.845390320s) e118/118) -- 0x56131e0 con 0x3e4a000 2026-02-11T23:31:49.896+0000 7fac55170640 10 log is not dirty 2026-02-11T23:31:49.896+0000 7fac55170640 20 osd.0 op_wq(0) _process empty q, waiting 2026-02-11T23:31:50.021+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 26 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:50.022035+0000/98.207435608s send_stamp 98.207435608s delta_ub -4.648923874s) ==== 2033+0+0 (crc 0 0 0) 0x568f080 con 0x3f56000 2026-02-11T23:31:50.021+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 26 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:50.022035+0000/98.207435608s send_stamp 98.207435608s delta_ub -4.648923874s) ==== 2033+0+0 (crc 0 0 0) 0x568ed80 con 0x3f56400 2026-02-11T23:31:50.021+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649123669s,-4.648923874s]) 2026-02-11T23:31:50.021+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:50.022035+0000/98.207435608s send_stamp 102.856559753s delta_ub 4.649123669s) -- 0x568ef00 con 0x3f56000 2026-02-11T23:31:50.021+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:50.021+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:50.021+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:50.021+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:50.021+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649153233s,-4.648923874s]) 2026-02-11T23:31:50.021+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:50.022035+0000/98.207435608s send_stamp 102.856582642s delta_ub 4.649153233s) -- 0x568f200 con 0x3f56400 2026-02-11T23:31:50.021+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:50.021+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:50.021+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:50.021+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:50.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:50.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:50.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:20.172618+0000) 2026-02-11T23:31:50.171+0000 7fac616e6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-11T23:31:50.260+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:50.260+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:50.260+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:50.260+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:50.260+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:50.260+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.047 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:50.260+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:50.261, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:50.260+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:50.260+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:50.260+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:50.260+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:50.260+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:50.869+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:50.869+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:51.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:51.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:51.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:21.172709+0000) 2026-02-11T23:31:51.297+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:51.297+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:51.297+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:51.297+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.047 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:51.297+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:51.299, recovery is active?:false restrictions:<....> 2026-02-11T23:31:51.297+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:51.297+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:51.297+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:51.297+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:51.297+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:51.720+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 27 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:51.722263+0000/99.907661438s send_stamp 99.907661438s delta_ub -4.648952007s) ==== 2033+0+0 (crc 0 0 0) 0x568f200 con 0x3f56400 2026-02-11T23:31:51.720+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649122238s,-4.648952007s]) 2026-02-11T23:31:51.720+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 27 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:51.722263+0000/99.907661438s send_stamp 99.907661438s delta_ub -4.648952007s) ==== 2033+0+0 (crc 0 0 0) 0x568ef00 con 0x3f56000 2026-02-11T23:31:51.720+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:51.722263+0000/99.907661438s send_stamp 104.556777954s delta_ub 4.649122238s) -- 0x568ed80 con 0x3f56400 2026-02-11T23:31:51.720+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:51.720+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:51.720+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:51.720+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:51.720+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649147511s,-4.648952007s]) 2026-02-11T23:31:51.720+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:51.722263+0000/99.907661438s send_stamp 104.556808472s delta_ub 4.649147511s) -- 0x568f080 con 0x3f56000 2026-02-11T23:31:51.720+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:51.720+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:51.721+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:51.721+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:51.899+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:51.900+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:51.953+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 34 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:51.954640+0000/102.808204651s send_stamp 102.808204651s delta_ub -1.980813742s) ==== 2033+0+0 (crc 0 0 0) 0x3da3980 con 0x396a400 2026-02-11T23:31:51.953+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 34 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:51.954640+0000/102.808204651s send_stamp 102.808204651s delta_ub -1.980813742s) ==== 2033+0+0 (crc 0 0 0) 0x568f080 con 0x396a000 2026-02-11T23:31:51.953+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980945230s,-1.980813742s]) 2026-02-11T23:31:51.953+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:51.954640+0000/102.808204651s send_stamp 104.789146423s delta_ub 1.980945230s) -- 0x3da2600 con 0x396a400 2026-02-11T23:31:51.953+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:51.953+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:51.953+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:51.953+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:51.953+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980968952s,-1.980813742s]) 2026-02-11T23:31:51.953+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:51.954640+0000/102.808204651s send_stamp 104.789169312s delta_ub 1.980968952s) -- 0x568ef00 con 0x396a000 2026-02-11T23:31:51.953+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:51.953+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:51.953+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:51.953+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:52.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:52.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:52.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:22.172728+0000) 2026-02-11T23:31:52.248+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:52.248+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:52.248+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:52.248+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.047 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:52.248+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:52.250, recovery is active?:false restrictions:<....> 2026-02-11T23:31:52.248+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:52.248+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:52.248+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:52.248+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:52.248+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:52.373+0000 7fac4d160640 5 osd.0 118 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x537bf3/0x5de000, compress 0x0/0x0/0x0, omap 0xc287, meta 0x1a43d79), peers [1,2,3,4] op hist []) 2026-02-11T23:31:52.373+0000 7fac4d160640 20 osd.0 118 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:52.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 105.208953857s delta_ub 1.980968952s) -- 0x568f380 con 0x396b400 2026-02-11T23:31:52.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 105.208953857s delta_ub 1.980968952s) -- 0x568f500 con 0x3752400 2026-02-11T23:31:52.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 105.208953857s delta_ub 4.649147511s) -- 0x568f680 con 0x396b800 2026-02-11T23:31:52.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 105.208953857s delta_ub 4.649147511s) -- 0x568f800 con 0x34c1000 2026-02-11T23:31:52.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 105.208953857s delta_ub 7.310453415s) -- 0x568f980 con 0x3740000 2026-02-11T23:31:52.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 105.208953857s delta_ub 7.310453415s) -- 0x568fb00 con 0x3740400 2026-02-11T23:31:52.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 105.208953857s delta_ub 9.886998177s) -- 0x568fc80 con 0x3740800 2026-02-11T23:31:52.373+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 105.208953857s delta_ub 9.886998177s) -- 0x568fe00 con 0x3740c00 2026-02-11T23:31:52.373+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 33 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 103.228202820s delta_ub -1.980754375s) ==== 2033+0+0 (crc 0 0 0) 0x568fe00 con 0x396b400 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:52.373+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980911016s,-1.980754375s]) 2026-02-11T23:31:52.373+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 30 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 97.898704529s delta_ub -7.310256481s) ==== 2033+0+0 (crc 0 0 0) 0x568fc80 con 0x3740400 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:52.373+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 33 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 103.228225708s delta_ub -1.980729461s) ==== 2033+0+0 (crc 0 0 0) 0x568f800 con 0x3752400 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:52.373+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 30 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 97.898674011s delta_ub -7.310278893s) ==== 2033+0+0 (crc 0 0 0) 0x568fe00 con 0x3740000 2026-02-11T23:31:52.373+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310432434s,-7.310256481s]) 2026-02-11T23:31:52.373+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:52.373+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:52.373+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:52.373+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:52.373+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980915546s,-1.980729461s]) 2026-02-11T23:31:52.373+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 30 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 95.322189331s delta_ub -9.886773109s) ==== 2033+0+0 (crc 0 0 0) 0x568fc80 con 0x3740800 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:52.373+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.886970520s,-9.886773109s]) 2026-02-11T23:31:52.373+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 32 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 100.560058594s delta_ub -4.648897648s) ==== 2033+0+0 (crc 0 0 0) 0x568f800 con 0x34c1000 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:52.373+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310484409s,-7.310278893s]) 2026-02-11T23:31:52.373+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 32 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 100.560081482s delta_ub -4.648870945s) ==== 2033+0+0 (crc 0 0 0) 0x568fc80 con 0x396b800 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:52.373+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:52.373+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649082184s,-4.648870945s]) 2026-02-11T23:31:52.373+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 30 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:52.374541+0000/105.208953857s send_stamp 95.322158813s delta_ub -9.886797905s) ==== 2033+0+0 (crc 0 0 0) 0x568fe00 con 0x3740c00 2026-02-11T23:31:52.373+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:52.373+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:52.373+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:52.373+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:52.373+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649114609s,-4.648897648s]) 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:52.373+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:52.373+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887022018s,-9.886797905s]) 2026-02-11T23:31:52.883+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.3 127.0.0.1:0/1114810440 30 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:52.884389+0000/98.408424377s send_stamp 98.408424377s delta_ub -7.310256481s) ==== 2033+0+0 (crc 0 0 0) 0x568fc80 con 0x3f56c00 2026-02-11T23:31:52.883+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310501575s,-7.310256481s]) 2026-02-11T23:31:52.883+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:52.884389+0000/98.408424377s send_stamp 105.718925476s delta_ub 7.310501575s) -- 0x568fb00 con 0x3f56c00 2026-02-11T23:31:52.883+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:52.883+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:52.883+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:52.883+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:52.883+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.3 127.0.0.1:0/1114810440 30 ==== osd_ping(ping e118 up_from 17 ping_stamp 2026-02-11T23:31:52.884389+0000/98.408424377s send_stamp 98.408424377s delta_ub -7.310256481s) ==== 2033+0+0 (crc 0 0 0) 0x568f800 con 0x3f56800 2026-02-11T23:31:52.883+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310545921s,-7.310256481s]) 2026-02-11T23:31:52.883+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/1114810440 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:52.884389+0000/98.408424377s send_stamp 105.718971252s delta_ub 7.310545921s) -- 0x568f500 con 0x3f56800 2026-02-11T23:31:52.883+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:52.883+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:52.883+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:52.883+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:52.893+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> conn(0x3e4a800 0x46e3600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).accept 2026-02-11T23:31:52.893+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> conn(0x3e4a800 0x46e3600 unknown :-1 s=BANNER_ACCEPTING pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-11T23:31:52.893+0000 7fac71bfd640 10 osd.0 118 ms_handle_fast_authentication new session 0x5678540 con 0x3e4a800 entity client.admin addr 2026-02-11T23:31:52.893+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/3236556566 conn(0x3e4a800 0x46e3600 crc :-1 s=READY pgs=3 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=client.7961 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-11T23:31:52.893+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.7961 127.0.0.1:0/3236556566 1 ==== command(tid 1: {"prefix": "get_command_descriptions"}) ==== 62+0+0 (crc 0 0 0) 0x25d9340 con 0x3e4a800 2026-02-11T23:31:52.893+0000 7fac626e8640 20 osd.0 118 OSD::ms_dispatch: command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:31:52.893+0000 7fac626e8640 20 osd.0 118 _dispatch 0x25d9340 command(tid 1: {"prefix": "get_command_descriptions"}) 2026-02-11T23:31:52.893+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/3236556566 -- command_reply(tid 1: 0 ) -- 0x25d94a0 con 0x3e4a800 2026-02-11T23:31:52.901+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.7961 127.0.0.1:0/3236556566 2 ==== command(tid 2: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x25d94a0 con 0x3e4a800 2026-02-11T23:31:52.901+0000 7fac626e8640 20 osd.0 118 OSD::ms_dispatch: command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:52.901+0000 7fac626e8640 20 osd.0 118 _dispatch 0x25d94a0 command(tid 2: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:52.901+0000 7fac713fc640 15 osd.0 118 collect_pg_stats 2026-02-11T23:31:52.901+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836508 v 0) -- 0x4f0f500 con 0x396a800 2026-02-11T23:31:52.901+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/3236556566 -- command_reply(tid 2: 0 ) -- 0x25d9340 con 0x3e4a800 2026-02-11T23:31:52.907+0000 7fac626e8640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] <== client.7961 127.0.0.1:0/3236556566 3 ==== command(tid 3: {"prefix": "flush_pg_stats"}) ==== 52+0+0 (crc 0 0 0) 0x25d9340 con 0x3e4a800 2026-02-11T23:31:52.907+0000 7fac626e8640 20 osd.0 118 OSD::ms_dispatch: command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:52.907+0000 7fac626e8640 20 osd.0 118 _dispatch 0x25d9340 command(tid 3: {"prefix": "flush_pg_stats"}) 2026-02-11T23:31:52.907+0000 7fac713fc640 15 osd.0 118 collect_pg_stats 2026-02-11T23:31:52.907+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836509 v 0) -- 0x4f0f200 con 0x396a800 2026-02-11T23:31:52.907+0000 7fac713fc640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> 127.0.0.1:0/3236556566 -- command_reply(tid 3: 0 ) -- 0x25d94a0 con 0x3e4a800 2026-02-11T23:31:52.907+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/3236556566 conn(0x3e4a800 msgr2=0x46e3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 79 2026-02-11T23:31:52.907+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/3236556566 conn(0x3e4a800 msgr2=0x46e3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-11T23:31:52.907+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/3236556566 conn(0x3e4a800 0x46e3600 crc :-1 s=READY pgs=3 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-11T23:31:52.907+0000 7fac71bfd640 1 --2- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] >> 127.0.0.1:0/3236556566 conn(0x3e4a800 0x46e3600 crc :-1 s=READY pgs=3 gs=6 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-11T23:31:52.907+0000 7fac626e8640 2 osd.0 118 ms_handle_reset con 0x3e4a800 session 0x5678540 2026-02-11T23:31:52.945+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:52.945+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:53.014+0000 7fac60ee5640 20 osd.0 118 reports for 0 queries 2026-02-11T23:31:53.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- mgrreport(unknown.0 +0-0 packed 3294 daemon_metrics=2) -- 0x4ff1180 con 0x396a800 2026-02-11T23:31:53.014+0000 7fac60ee5640 15 osd.0 118 collect_pg_stats 2026-02-11T23:31:53.014+0000 7fac60ee5640 1 -- [v2:127.0.0.1:6800/460666604,v1:127.0.0.1:6801/460666604] --> [v2:127.0.0.1:6808/954425105,v1:127.0.0.1:6809/954425105] -- pg_stats(0 pgs seq 21474836510 v 0) -- 0x567ef00 con 0x396a800 2026-02-11T23:31:53.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:53.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:53.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:23.172801+0000) 2026-02-11T23:31:53.279+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:53.279+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:53.279+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:53.279+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:53.279+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:53.279+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.047 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:53.279+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:53.280, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:53.279+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:53.279+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:53.279+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:53.279+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:53.279+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:53.935+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:53.935+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:54.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:54.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:54.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:24.172875+0000) 2026-02-11T23:31:54.232+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:54.232+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:54.232+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_random_backoff: lost coin flip, randomly backing off (ratio: 0.660) 2026-02-11T23:31:54.232+0000 7fac683ea640 15 osd.0 osd-scrub:restrictions_on_scrubbing: Lost on the dice. Regular scheduled scrubs are not permitted. 2026-02-11T23:31:54.232+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:54.232+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.047 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:54.232+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:54.234, recovery is active?:false restrictions:<.backoff...> 2026-02-11T23:31:54.232+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:54.232+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:54.232+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:54.232+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:54.232+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:54.259+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.4 127.0.0.1:0/864727417 30 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:54.260865+0000/97.208366394s send_stamp 97.208366394s delta_ub -9.886773109s) ==== 2033+0+0 (crc 0 0 0) 0x568fe00 con 0x3f57400 2026-02-11T23:31:54.259+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.4 127.0.0.1:0/864727417 30 ==== osd_ping(ping e118 up_from 21 ping_stamp 2026-02-11T23:31:54.260865+0000/97.208366394s send_stamp 97.208366394s delta_ub -9.886773109s) ==== 2033+0+0 (crc 0 0 0) 0x568f500 con 0x3f57800 2026-02-11T23:31:54.259+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887030602s,-9.886773109s]) 2026-02-11T23:31:54.259+0000 7fac703fa640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:54.260865+0000/97.208366394s send_stamp 107.095397949s delta_ub 9.887030602s) -- 0x568f980 con 0x3f57400 2026-02-11T23:31:54.259+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:54.259+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:54.259+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:54.259+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:54.259+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887068748s,-9.886773109s]) 2026-02-11T23:31:54.259+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/864727417 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:54.260865+0000/97.208366394s send_stamp 107.095443726s delta_ub 9.887068748s) -- 0x568f800 con 0x3f57800 2026-02-11T23:31:54.259+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:54.259+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:54.259+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:54.259+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:54.919+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:54.919+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:55.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:55.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:55.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:25.172943+0000) 2026-02-11T23:31:55.184+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:55.184+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:55.184+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:55.184+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.047 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:55.184+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:55.185, recovery is active?:false restrictions:<....> 2026-02-11T23:31:55.184+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:55.184+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:55.184+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:55.184+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:55.184+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:55.273+0000 7fac4d160640 5 osd.0 118 heartbeat osd_stat(store_statfs(0x0/0x0/0x384000, data 0x537bf3/0x5de000, compress 0x0/0x0/0x0, omap 0xc287, meta 0x1a43d79), peers [1,2,3,4] op hist []) 2026-02-11T23:31:55.273+0000 7fac4d160640 20 osd.0 118 check_full_status cur ratio 1, physical ratio 1, new state failsafe 2026-02-11T23:31:55.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6816/4244365097,v1:127.0.0.1:6817/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 108.109107971s delta_ub 1.980915546s) -- 0x568e000 con 0x396b400 2026-02-11T23:31:55.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6814/4244365097,v1:127.0.0.1:6815/4244365097] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 108.109107971s delta_ub 1.980915546s) -- 0x568e900 con 0x3752400 2026-02-11T23:31:55.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6824/2662569638,v1:127.0.0.1:6825/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 108.109107971s delta_ub 4.649114609s) -- 0x568ec00 con 0x396b800 2026-02-11T23:31:55.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6822/2662569638,v1:127.0.0.1:6823/2662569638] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 108.109107971s delta_ub 4.649114609s) -- 0x567c900 con 0x34c1000 2026-02-11T23:31:55.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6832/1114810440,v1:127.0.0.1:6833/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 108.109107971s delta_ub 7.310545921s) -- 0x567c480 con 0x3740000 2026-02-11T23:31:55.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6830/1114810440,v1:127.0.0.1:6831/1114810440] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 108.109107971s delta_ub 7.310545921s) -- 0x567d500 con 0x3740400 2026-02-11T23:31:55.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6840/864727417,v1:127.0.0.1:6841/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 108.109107971s delta_ub 9.887068748s) -- 0x3e02d80 con 0x3740800 2026-02-11T23:31:55.273+0000 7fac4d160640 1 -- 127.0.0.1:0/460666604 --> [v2:127.0.0.1:6838/864727417,v1:127.0.0.1:6839/864727417] -- osd_ping(ping e118 up_from 5 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 108.109107971s delta_ub 9.887068748s) -- 0x3e03500 con 0x3740c00 2026-02-11T23:31:55.273+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6832/1114810440 31 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 100.798858643s delta_ub -7.310256958s) ==== 2033+0+0 (crc 0 0 0) 0x3e03500 con 0x3740000 2026-02-11T23:31:55.273+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6824/2662569638 33 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 103.460205078s delta_ub -4.648905277s) ==== 2033+0+0 (crc 0 0 0) 0x3e02d80 con 0x396b800 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:55.273+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310437679s,-7.310256958s]) 2026-02-11T23:31:55.273+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6814/4244365097 34 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 106.128372192s delta_ub -1.980739713s) ==== 2033+0+0 (crc 0 0 0) 0x567c900 con 0x3752400 2026-02-11T23:31:55.273+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:55.273+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:55.273+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:55.273+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:55.273+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980930448s,-1.980739713s]) 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:55.273+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6838/864727417 31 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 98.222343445s delta_ub -9.886772156s) ==== 2033+0+0 (crc 0 0 0) 0x3e03500 con 0x3740c00 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:55.273+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649107456s,-4.648905277s]) 2026-02-11T23:31:55.273+0000 7fac71bfd640 1 -- 127.0.0.1:0/460666604 <== osd.2 v2:127.0.0.1:6822/2662569638 33 ==== osd_ping(ping_reply e118 up_from 13 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 103.460235596s delta_ub -4.648879051s) ==== 2033+0+0 (crc 0 0 0) 0x567c900 con 0x34c1000 2026-02-11T23:31:55.273+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:55.273+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:55.273+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:55.273+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:55.273+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.4 v2:127.0.0.1:6840/864727417 31 ==== osd_ping(ping_reply e118 up_from 21 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 98.222366333s delta_ub -9.886748314s) ==== 2033+0+0 (crc 0 0 0) 0x3e02d80 con 0x3740800 2026-02-11T23:31:55.273+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649089336s,-4.648879051s]) 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:55.273+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887016296s,-9.886772156s]) 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.4 from_epoch 118 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:55.273+0000 7fac703fa640 1 -- 127.0.0.1:0/460666604 <== osd.1 v2:127.0.0.1:6816/4244365097 34 ==== osd_ping(ping_reply e118 up_from 9 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 106.128410339s delta_ub -1.980705619s) ==== 2033+0+0 (crc 0 0 0) 0x3e03500 con 0x396b400 2026-02-11T23:31:55.273+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6836/864727417 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.4 up_from 21 peer_clock_delta [-9.887001991s,-9.886748314s]) 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:55.273+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:55.273+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980969906s,-1.980705619s]) 2026-02-11T23:31:55.273+0000 7fac723fe640 1 -- 127.0.0.1:0/460666604 <== osd.3 v2:127.0.0.1:6830/1114810440 31 ==== osd_ping(ping_reply e118 up_from 17 ping_stamp 2026-02-11T23:31:55.274696+0000/108.109107971s send_stamp 100.798881531s delta_ub -7.310232639s) ==== 2033+0+0 (crc 0 0 0) 0x3e02d80 con 0x3740400 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.3 from_epoch 118 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:55.273+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6828/1114810440 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:55.273+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.3 up_from 17 peer_clock_delta [-7.310503483s,-7.310232639s]) 2026-02-11T23:31:55.955+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:55.955+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000 2026-02-11T23:31:56.053+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.1 127.0.0.1:0/4244365097 35 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:56.054807+0000/106.908370972s send_stamp 106.908370972s delta_ub -1.980705619s) ==== 2033+0+0 (crc 0 0 0) 0x567c900 con 0x396a400 2026-02-11T23:31:56.053+0000 7fac71bfd640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980944991s,-1.980705619s]) 2026-02-11T23:31:56.053+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.1 127.0.0.1:0/4244365097 35 ==== osd_ping(ping e118 up_from 9 ping_stamp 2026-02-11T23:31:56.054807+0000/106.908370972s send_stamp 106.908370972s delta_ub -1.980705619s) ==== 2033+0+0 (crc 0 0 0) 0x3e02d80 con 0x396a000 2026-02-11T23:31:56.053+0000 7fac71bfd640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:56.054807+0000/106.908370972s send_stamp 108.889320374s delta_ub 1.980944991s) -- 0x568e900 con 0x396a400 2026-02-11T23:31:56.053+0000 7fac71bfd640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:56.053+0000 7fac71bfd640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:56.053+0000 7fac71bfd640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:56.053+0000 7fac71bfd640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:56.053+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.1 up_from 9 peer_clock_delta [-1.980967522s,-1.980705619s]) 2026-02-11T23:31:56.053+0000 7fac723fe640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/4244365097 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:56.054807+0000/106.908370972s send_stamp 108.889343262s delta_ub 1.980967522s) -- 0x567d500 con 0x396a000 2026-02-11T23:31:56.053+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.1 from_epoch 118 2026-02-11T23:31:56.053+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:56.053+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:56.053+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6812/4244365097 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:56.171+0000 7fac616e6640 10 monclient: tick 2026-02-11T23:31:56.171+0000 7fac616e6640 10 monclient: _check_auth_tickets 2026-02-11T23:31:56.171+0000 7fac616e6640 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2026-02-11T23:31:26.173015+0000) 2026-02-11T23:31:56.227+0000 7fac683ea640 10 osd.0 118 tick_without_osd_lock 2026-02-11T23:31:56.227+0000 7fac683ea640 0 osd.0 118 set_statfs fake total 3686400 adjust available 0 2026-02-11T23:31:56.227+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_time_permit: should run between 0 - 0 now 23 = yes 2026-02-11T23:31:56.227+0000 7fac683ea640 20 osd.0 osd-scrub:scrub_load_below_threshold: loadavg per cpu 0.047 < max 2000.000 (#CPUs:24) = yes 2026-02-11T23:31:56.227+0000 7fac683ea640 10 osd.0 osd-scrub:initiate_scrub: scrub scheduling (@tick) starts. time now:2026-02-11T23:31:56.228, recovery is active?:false restrictions:<....> 2026-02-11T23:31:56.227+0000 7fac683ea640 20 osd.0 osd-scrub:initiate_scrub: no PGs are ready for scrubbing 2026-02-11T23:31:56.227+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate 0 attempts, promoted 0 objects and 0 B; target 25 obj/sec or 5 MiB/sec 2026-02-11T23:31:56.227+0000 7fac683ea640 20 osd.0 118 promote_throttle_recalibrate new_prob 1000 2026-02-11T23:31:56.227+0000 7fac683ea640 10 osd.0 118 promote_throttle_recalibrate actual 0, actual/prob ratio 1, adjusted new_prob 1000, prob 1000 -> 1000 2026-02-11T23:31:56.227+0000 7fac683ea640 20 osd.0 118 maybe_send_beacon beacon would be too frequent; skipping 2026-02-11T23:31:56.421+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 28 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:56.422468+0000/104.607864380s send_stamp 104.607864380s delta_ub -4.648879051s) ==== 2033+0+0 (crc 0 0 0) 0x3e03500 con 0x3f56400 2026-02-11T23:31:56.421+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 28 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:56.422468+0000/104.607864380s send_stamp 104.607864380s delta_ub -4.648879051s) ==== 2033+0+0 (crc 0 0 0) 0x567d500 con 0x3f56000 2026-02-11T23:31:56.421+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649122238s,-4.648879051s]) 2026-02-11T23:31:56.421+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:56.422468+0000/104.607864380s send_stamp 109.256980896s delta_ub 4.649122238s) -- 0x567c480 con 0x3f56400 2026-02-11T23:31:56.421+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:56.421+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:56.421+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:56.421+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:56.421+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649147034s,-4.648879051s]) 2026-02-11T23:31:56.421+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:56.422468+0000/104.607864380s send_stamp 109.257003784s delta_ub 4.649147034s) -- 0x3e02d80 con 0x3f56000 2026-02-11T23:31:56.421+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:56.421+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:56.421+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:56.421+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:56.921+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] <== osd.2 127.0.0.1:0/2662569638 29 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:56.922650+0000/105.108047485s send_stamp 105.108047485s delta_ub -4.648933411s) ==== 2033+0+0 (crc 0 0 0) 0x567c480 con 0x3f56400 2026-02-11T23:31:56.921+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] <== osd.2 127.0.0.1:0/2662569638 29 ==== osd_ping(ping e118 up_from 13 ping_stamp 2026-02-11T23:31:56.922650+0000/105.108047485s send_stamp 105.108047485s delta_ub -4.648933411s) ==== 2033+0+0 (crc 0 0 0) 0x3e02d80 con 0x3f56000 2026-02-11T23:31:56.921+0000 7fac703fa640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649127960s,-4.648933411s]) 2026-02-11T23:31:56.921+0000 7fac703fa640 1 -- [v2:127.0.0.1:6804/460666604,v1:127.0.0.1:6805/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:56.922650+0000/105.108047485s send_stamp 109.757179260s delta_ub 4.649127960s) -- 0x3e03500 con 0x3f56400 2026-02-11T23:31:56.921+0000 7fac703fa640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:56.921+0000 7fac703fa640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:56.921+0000 7fac703fa640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:56.921+0000 7fac703fa640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:56.921+0000 7fac723fe640 20 osd.0 118 handle_osd_ping new stamps hbstamp(osd.2 up_from 13 peer_clock_delta [-4.649154663s,-4.648933411s]) 2026-02-11T23:31:56.921+0000 7fac723fe640 1 -- [v2:127.0.0.1:6806/460666604,v1:127.0.0.1:6807/460666604] --> 127.0.0.1:0/2662569638 -- osd_ping(ping_reply e118 up_from 5 ping_stamp 2026-02-11T23:31:56.922650+0000/105.108047485s send_stamp 109.757202148s delta_ub 4.649154663s) -- 0x567d500 con 0x3f56000 2026-02-11T23:31:56.921+0000 7fac723fe640 20 osd.0 118 get_con_osd_cluster to osd.2 from_epoch 118 2026-02-11T23:31:56.921+0000 7fac723fe640 20 osd.0 118 get_nextmap_reserved map_reservations: {118=1} 2026-02-11T23:31:56.921+0000 7fac723fe640 20 osd.0 118 release_map epoch: 118 2026-02-11T23:31:56.921+0000 7fac723fe640 10 osd.0 118 maybe_share_map: con v2:127.0.0.1:6820/2662569638 our osdmap epoch of 118 is not newer than session's projected_epoch of 118 2026-02-11T23:31:56.971+0000 7fac68beb640 10 osd.0 118 tick 2026-02-11T23:31:56.971+0000 7fac68beb640 20 osd.0 118 tick last_purged_snaps_scrub 2026-02-11T23:30:08.161323+0000 next 2026-02-13T09:24:16.598823+0000